Is it a bad idea to run voice and data over the same bonded T1?

Currently we have a T1 which we use for both our voice and data.
We are looking at getting a bonded T1 but have been told by the provider that it would be a bad idea to run both voice and data on a bonded T1 (Multilink PPP) and that we should either:

A) Put in a bonded T1 for data only and have a 2nd non-bonded T1 for voice, or
B) Put in a 2nd T1 for data only and make the current T1 voice only

We don't use the voice as much and the reason we looked into a bonded T1 is for more internet speed, but obviously we still need to keep voice services with good quality and reliability. Unfortunately T1 is the only thing available to us since all the lines here are copper.

I am not familiar with bonded T1s. Tried googling about bonded T1s and having both data and voice on it but couldn't come up with anything helpful. So my question is, will we lose service quality by getting a bonded T1 and putting both voice and data on it?

From serverfault Nate Pinchot

• By voice, do you mean 1.) you're using a MUX to split channels out of the current T1 for voice, or 2.) you've got VOIP as part of your data traffic?

  1. I don't think there would be any difference whether your link was 1 T1 or a MPPP link, as long as all of your MUX and termination equipment supported MPPP. You'd have to confirm that it would work though, so that may be what the mean. Maybe your existing equipment doesn't and they're trying to save you the cost of an upgrade.

  2. If it's all data (VOIP is just data) then it doesn't make a lick of difference what the link is. As long as you're giving priority to the VOIP, you could be using T1, T3, dial-up, cable modems, ham radio, ISDN, carrier pigeons, smoke signals - you just want to give priority to VOIP and ensure that you don't fill the pipe. Of course, you do also need to concern yourself with latency, jitter, etc - but those factors are independent of what the pipe is (although some will be less suitable.)

  Nate Pinchot : Unfortunately I don't know the exact answer to what voice means in this scenario since it is managed by the provider. They send it across the wire and then we have X amount of standard analog lines that we plug into our PBX. I suspect the correct answer is #2 and they probably just want us to spend more money. I spoke with the service provider a few minutes ago and they said they could make the scenario work but they wouldn't guarantee voice quality. Understandably the business group doesn't like that answer so we are probably sticking with a single T1 for now.

  mfinni : In scenario 2, I don't see how you could double the available bandwidth and have worse performance compared to what you have now. Are you having problems now? If not, then I don't see how more bandwidth could hurt you - assuming you don't anticipate an increase in voice usage. If you are seeing problems, then more bandwidth would help, not hurt, regardless. It might not help as much as separating them into different links, but proper QOS would help as well and you can do that on any link.

  mfinni : Get more details from the provider. Don't let them snow you.

  Evan Anderson : +1 - VoIP over smoke signals... heh heh.
  From mfinni

Is there a vmware appliance for software development?

I use Assembla.com and it does everything I need, SVN, bug tracking, ticketing... the only thing I don't like is that the server is not mine, I'm putting all my company property on a server in the coulds. I was wondering if there is an virtual machine to download and put on my server to serve all Assembla does?

From serverfault Ali Shafai

• Hmmm, i can't reach assembla.com right now - seems to be down =)

  I don't know of a VM that contains all these tools, but i'm sure it'll be easy to set one up really fast.

  Another question: Do you think it's such a good idea? I would pick an old pentium 3 from ebay and set it up as dev-server (but remember to make backups, and backups and BACKUPS!).

  Otherwise you'd have a vm running all the time without actually using it all the time. And even a P3 (or whatever you like) with a fast disk or raid controller may be faster than your VM.

  Alternative: rent a virtual Server (Here in Europe: 3 GHz, 1024MB RAM, 30GB disk - 8€/Month).

  Ali Shafai : Main point about assembla is all those tools are connected. there is a lot of setting up I can save if there is a vm image ready. and I already have a VMware server machine and all I have to do is to add this one. 8€/Month sounds great, which company is this?

  lajuette : netcup - a company from karlsruhe/germany. But there are a lot of other comanies with almost equal prices an maybe virtualization technologies better suited for your needs. Hav you searched the VMWare Appliance Marketplace? http://www.vmware.com/appliances/

  Chris Thorpe : Bleh to setting up a cheap old PC as a dev server. Horrible idea. These things are the bane of IT supports life when you're 5 years down the line and you find what has become a 'critical production' server crammed under someone's desk. If you already run a virtual environment within your organisation, VMs are absolutely the way to go in this kind of scenario.

  lajuette : Ali didn't say in what kind of environment he will be using this server. And do you think it's better to run a VM of a dev server instead of a real server? He didn't mention the existing VM host in the first place either. Compared to a service with unexplained downtimes an old machine seems a very good idea to me. A modern and stable (virtual-)machine is the best way to go if you're setting up a dev server in a company. But if you don't have the resources or need the server for yourself a small machine will/has to be enough. And i'm not talking about cheap, but reliable old hardware =)

  Chris Thorpe : Points taken. I wouldn't agree in any professional scenario, but if he's doing home dev with little budget, an old server would be a viable option.
  From lajuette

• You may want to have a look around the turnkey linux website. When I was looking for an easy way to evaluate an integrated bug tracking/SCM server I found the TRAC and Redmine appliances interesting, if not quite what we needed at the time. Your mileage may vary however.

  Oh, and if you're not absolutely wedded to SVN, you may want to check out Mercurial for your SCM, you'll never want to go back to SVN after trying it. Distributed revision control is a breath of fresh air after years of 'tranditional' source control and if you use TortoiseSVN, then TortoiseHg will be a breeze.

  From Mark Booth

Can i safely block all ports on my sever

I have a web server running pleks and get attacked a lot every day. I configured a firewall, and there are some predefined services such as www, and ftp, etc.

I am planning to block all the ports except for the www port and the PLESK port. When I need ftp or ssh access for example i will open the ports in PLESK and then start to work.

Is this a good thing to do or are there some downsides on doing this.

There are some ports I am not sure of, such as SMTP,POP3,IMAP, DNS. Can i close these ports or is there no need to do so.

From serverfault Saif Bechan

• You will probably be ok, but you might be better off leaving a hole for your IP, just in case something goes wrong. You might also want to make sure you don't block connections from 127.0.0.1, as that is the localhost IP and may be necessary for internal services to connect to themselves to keep things working properly (it depends on what type of stuff you have running on the system).

  As far as the ports you mention, here is what those are, you can close them if you don't use them:

  SMTP: Email (Server to server or incoming) (needed to accept email if this server receives email for your domain)

  POP3: Email clients (needed if this server has email clients that connect to it)

  IMAP: Email clients (needed if this server has email clients that connect to it)

  DNS: Domain Name services (needed if this server acts is the primary for the domains it hosts)

  Good luck,

  --jed

  Saif Bechan : Thank you for the quick response. This was the exactly my guess but i asked just to be sure. I have services running so my internal ports are not blocked. One question tho. "leaving a hole for your IP". Do you mean the ip of the server, or do you mean the remote system I use to log in to the server. The IP's of the remote machines I use are always different, so that is no option for me. And can you explain the point of leaving a hole for the server itself. Can the server connect to itself?

  Jed Daniels : I was referring to the IP of the system you are connecting from at the time you block all the ports. Just in case something goes wrong when you make the edits you want to be able to get back in to fix them. Yes, servers can connect to themselves, and many do (for example, if you have a web server running an application that uses mysql or some other database, the web-server will access the database over a network connection to and from 127.0.0.1). This is why I also recommend making sure you don't accidentally cut off this type of traffic. Cheers, --jed
  From Jed Daniels

• A prudent approach would be to deny all connections by default, and only open up ports when they are demonstrated to be needed for some valid purpose.

  Beware of being too unresponsive to user requests, though: make sure that when any user asks for a port to be opened that the user's request is heeded promptly, visibly, and seriously for all users to see, otherwise you'll just end up with a userbase working around the block list by tunnelling through the ports you do open.

  From bignose

• why not open ssh for your IP range only.. that way if plesk crashes you aren't locked out..

  Plus you can use ssh keys to make it even more secure and deny password logins.

  From Mike

How to properly remove disk from PERC 6/i RAID controller ?

I have a Dell T710, coming with PERC 6/i RAID controller. The current raid has 2x500 GB hard drives (with the OS), and 6x1000 GB hard drives (in RAID-6, currently empty). I would like to take one 1000 GB disk physically out to keep as an immediate spare in case of a crash, and configure the remaining 5x1000 GB in a single VD RAID-6.

This is all nice and clean and works, until I realized that the display on the machine reports the lack of the 8th disk as an error. It's marked as error, but appears to be a warning, since the machine is fully functional.

My question is: what is the best way to keep one disk as a spare out of the array? should I disassemble the disk from the cradle and insert the empty cradle in the array ? Or should I just silence the error in the display in some way (how?). I know that what I am doing sounds pretty strange, but here is academia and having a spare disk available could take weeks. Better to have one ready in my drawer for any emergency.

From serverfault Stefano Borini

• Putting that spare disk in your drawer doesn't make too much sense. Instead, leave it in the server and mark it as a hot spare. Instructions here. Then, you'll have a OS mirror and a RAID5 array for data and if any of those disks fails, the controller will automatically rebuild with the hot spare.

  As for why you are getting an error about the last disk, you'll have to provide more detail. What is the error? Have you already rebuilt all of the RAID sets so that disk is not currently configured in a RAID set?

  Stefano Borini : I see two problems with the hot spare. The first is that it makes the setup too complex for the environment I'm in (if someone has to put the hands on the server). The second is that I want effectively a cold spare to prevent any type of stress (electric or mechanic) on it. I am rebuilding the array right now. It claims "E1812" on the display, about the missing disk. I assume it's probably a good idea to go for a hot spare as you said, but I've never used this kind of medium irons, so I am quite improvising here. Any more hint very welcome on this regard.

  Insyte : Yeah, you're overthinking this. Hot spares are good. Cold spares are bad. Take icky2000's advice and mark your extra drive as a hot spare. It's a simple configuration, far simpler than expecting someone unfamiliar with your config to get the RAID array to properly rebuild onto a new drive. Then when a drive *does* fail, they have an immediate safety net until they can figure out how to replace a drive.

  : E1812 is an information only "error" - just means no disk is there. Just rebuild the RAID set without it, acknowledge the "alert" and proceed. I understand that you don't have experience with server hardware but you're shooting down good advice with reasons that don't make sense. Put the disk back in and make it a hot spare.
  From

IIS not listening over external network, all other traffic working

Hello there,

I have a very odd situation, I have a server (let's call it X) running 2008 R2 with two NIC's in it, one is connected to the work domain and has a subnet of 192.168.10.0/24 the other is connected to a ADSL connection and has a subnet of 192.168.1.0/24. The server has IIS installed.

On the ADSL connection I have setup a dynamic dns and port forwarding to allow external HTTP, HTTPS, FTP and RDP connections. FTP and RDP are working fine however neither HTTP or HTTPS are working at all.

I can browse the websites by going to localhost on the machine, the HTTP and HTTPS ports appear as "Filtered" when I try to scan them using PortQueryUI and browsers respond with a "Server took too long to load or was not responding" error.

This was working fine just a few days ago, Windows firewall is disabled I don't have any software firewall on it. And I'm really lost.

Any help would be great.

From serverfault Beuy

• Can you try assigning the external IP to IIS and then try accessing the Web site using http://ip_address. Let's see if that works. If that fails, port 80 is blocked on that ip. http://localhost will always. Try netstat -ano and check if you are actually getting connected on port 80 to your external ip.

  Beuy : Hi Vivek thanks for the response, I'm not 100% certain I understand what you are saying but here are my results: 1. Bind IP to external IP 2. iisreset 3. http://externalipaddress = Failed server taking too long to respond 4. http://localhost = Success 5.netstat -ano TCP 0.0.0.0:80 0.0.0.0.0 LISTENING 4 (PID 4 is system) not sure what the deal with that is. This server also has MS SQL2008 and Reporting Services Installed.

  Vivek : when you did netstat -ano, did you see an entry of external ip corresponding to port 80? Basically, when you say "server taking long time to respond" means ultimately it will timeout. IIS is not getting the request and hence the timeout. I will also check IIS logfile to verify if the request actually reached IIS.

  Beuy : I didn't see any entries for the external IP. Which logfile should I be checking? (Sorry bit of an IIS newbie)

  Vivek : IIS logfiles are stored in C:\inetpub\logs\LogFiles. If this is your first site, a folder W3SVC1 will be created to store the logfile.

  Beuy : No sign of the external IP in the logfile.

  Vivek : so as thought, the request is not reaching IIS at all. You have the port blocked. check the firewall again. Make sure you have HTTP enabled on the external NIC.

  Beuy : The thing is that I'm 99.99999% certain that the firewall is not blocking the request. When I do a port scan I get a bizarre situation, the first time around it lists it as being Filtered, the second time it's not listening. Every other port is working fine.

  Beuy : Found the problem, there was an IP address conflict and port 80 was going to a different machine, thanks for the help again Vivek

  Vivek : Nice you found the solution :-)
  From Vivek

Using systeminfo to get the OS Name

I need to find the flavor of Windows that is running using a batch file that will run on anything from Windows NT to Windows 7. I'm using a method based on this page with some minor changes.

Systeminfo gives the flavor of Windows that is running. Is there any authoritative list of names that can be returned? If so where would I find the list?

My intention is to do something like:

 winVer=Unknown

 rem NT doesn't have systeminfo
 ver | find "Windows NT" > nul
 if %errorlevel%==0 set winVer=WinNT

 if exist %SystemRoot%\system32\systeminfo.exe (
  for /f "delims=: tokens=2" %%v in ('sysinfo ^| find "OS Name"') do (
   set verStr=%%v
  )
  echo %verStr% | find "Windows XP" > nul
  if %errorlevel%==0 set winVer=WinXP
  echo %verStr% | find "Windows Vista" > nul
  if %errorlevel%==0 set winVer=WinVista
  ... etc
 )

Thanks

From serverfault WileCau

• Check this thread: http://stackoverflow.com/questions/1792740/how-to-tell-what-version-of-windows-and-or-cmd-exe-a-batch-file-is-running-on

  WileCau : @Sergey, thanks for the link. Using ver was my first thought but it isn't definitive or consistent enough, e.g. NT returns "Windows NT Version 4.0", XP returns "Microsoft Windows XP [Version 5.1.2600]", Server 2008 returns "Microsoft Windows [Version 6.1.7600]". Using the version number to infer the OS Name can result in ambiguity, e.g. the other thread has results like "Windows Vista or Windows Server 2008". The OS Name from systeminfo tells you exactly which one it is, without the need for interpretation. The link told me about %PROCESSOR_ARCHITECTURE% though, which I also need :) Thanks
  From Sergey

Bash script to create mass series of directories

I need to create a Bash script to go into every user's home folder, seek out a wp-content folder, create a directory uploads under it, and then chmod 0756 uploads.

How do I achieve this?

I imagine I need to use find with a regexp/regex, and then tell it to run another bash script on the results.

From serverfault ServerChecker

• Something like this should work (I haven't tested it)

  dirs=`find /home -type d -name "wp-content"` 
  
  for dir in $dirs; do
      if [ ! -e $dir/uploads ]; then 
          mkdir $dir/uploads
          chmod 0765 $dir/uploads
      fi
  done
  

  Dennis Williamson : That fails if there's a regular file named "uploads".

  ServerChecker : A space is required after the first bracket and before the next. Doublequotes are probably best around $dir/uploads. I only know this because I tested, got errors, and googled.

  ServerChecker : If we use dirs=$(locate -r 'wp-content$' | grep -i '/home'), it's probably going to run faster than find, I found out. The only catch is that one needs to ensure that updatedb has been run in the past 24 hours.

  Dennis Williamson : @Volomike: `locate` requires the filenames to be in a database that's updated by `updatedb` which is typically run once a day by `cron`. Files and directories that have been created since the last run will be missed by your script if you use `locate`.

  Dennis Williamson : Another problem here is that `find` is recursive and will find directories named "wp-content" below the level you intend. You should use the `-maxdepth` option.

  ServerChecker : In our case, locate is best because it reduces server load and because these wp-content dirs will have been created 4 days ago.

  theotherreceive : @Dennis Yes, you're right on the file called uploads, -e would have been better there. However, there's nothing specified in the question as to the max depth wp-content can reside in

  Dan Andreatta : Instead of using `find`, you can also try `for dir in /home/*/wp-content` ... or similar
  From theotherreceive

• The above answer is a better one, but here is a primitive, but functional, alternative:

  for i in user1 user2;do mkdir $i/wp-content;chmod 0765 $i/wp-content;done
  

  This assumes you are in the parent directory of all your users, and they are in the same directory.

  This will also fail if there is a file named "uploads", but will continue on.

  Good luck,

  --jed

  Dennis Williamson : You forgot to include the "uploads" directory in your command. You should use `&&` instead of `;` between the `mkdir` and the `chmod`. And what if there are hundreds (or more) users?

  Jed Daniels : All excellent points, although I might not use `&&` because if the directory already exists, I still might want to `chmod` it (not sure, the poster wasn't specific). If there are hundreds of users, I'd probably use the answer from the previous poster, which I mentioned is a better answer. But if there were just a few users, and I wanted to quickly get this out of the way without bothering to create a script, make it executable, then run it, I'd use my admittedly primitive one-liner. Thanks, --jed
  From Jed Daniels

PostgreSQL lots of large Arrays and Writes

Hi,

I am running a python program that spawns 8 threads and as each thread launch its own postmaster process via psycopg2. This is to maximize the use of my CPU-cores (8). Each thread call a series of SQL Functions. Most of these functions go through many thousands of rows each associated to a large FLOAT8[] Array (250-300) values by using unnest() and multiplying each FLOAT8 by an another FLOAT8 associated to each row. This Array approach minimized the size of the Indexes and the Tables. The Function ends with an Insert into another Table of a row of the same form (pk INT4, array FLOAT8[]). Some SQL Functions called by python will Update a row of these kind of Tables (with large Arrays).

Now I currently have configured PostgreSQL to use most of the memory for cache (effective_cache_size of 57 GB I think) and only a small amount of it for shared memory (1GB I think). First, I was wondering what the difference between Cache and Shared memory was in regards to PostgreSQL (and my application).

What I have noticed is that only about 20-40% of my total CPU processing power is used during the most Read intensive parts of the application (Select unnest(array) etc). So secondly, I was wondering what I could do to improve this so that 100% of the CPU is used. Based on my observations, it does not seem to have anything to do with python or its GIL.

Thanks

From serverfault

• It seems that you have hit an I/O bottleneck. You have a lot of cache memory, but how big is the dataset? What is the current disk configuration? How busy are the disks? Could the bottleneck be the network?

  Another thing to check is how much work memory has each process. It is possible that there is a lot of memory traffic for no reason.

  This site has a good overview for tuning postgres.

  : The dataset is more then 60GB, but the Tables whose entire rows are used by each SQL Function take about 15GB. This disk configuration is 2 RAID 10 Arrays of 4 SAS 10k Rpm 146 GB each. The 15 GB of frequently used Tables are on the first RAID 10 array, while everything else, including those than get a lot of Inserts, are on the other RAID 10 array. The disks do not seem that busy. Its definitely not the network. Each process's memory usage does seem to grow a lot as the application progresses. What do you suggest?

  Dan Andreatta : You should check the I/O performance anyway just to exclude it as an issue. Check mainly the number of trasactions per seconds. Second, each postgres worker should get a fixed amount of memory, specified by the work memory config option. Try increase that. Edited the answer as well

  : That working memory config seems to have helped. Thx.
  From Dan Andreatta

• effective_cache_size does not change any memory setting, it's used only for estimation purposes in query planning. Crank up the shared_buffers to about 25% of your available RAM and see if there are any differences in speed.

  Also, use EXPLAIN to get the queryplan and see if you need some extra indexes or better configuration.

  : Yup, the shared_buffers to 25% helped. Thx

  : Why not increase it to 50%?

  Frank Heikens : Because you don't want to kill your server. If you have a lot of RAM, you might use more than 25% for shared_buffers, but be carefull. Be very carefull! PostgreSQL is using more RAM than just shared_buffers and your server also needs RAM for other things. Every connection needs RAM, every sort operation needs RAM, etc. etc. etc. When you push it too much, your server might kill the postmaster (eats too much RAM) or starts swapping and killing performance. A better queryplan, using less RAM, might be a better idea. But that's up to the database.

  : I see. When I look at top or dstat, I see 58GB used for cache. Does this include both shared buffers and kernel cache? It seems like it is both because my 8 postmaster processes are using 9 GB of shared memory while I only have 64GB of ram. ? Thx again :)

  Frank Heikens : See this picture: http://www.postgresql.org/files/documentation/books/aw_pgsql/hw_performance/node3.html And yes, the 58GB is all the cache. PostgreSQL doesn't take all shared_buffers when it starts, it just takes what it needs, up to the limits set in the configuration.

  : I see. Nice link. Thanks

  : Yeah, I tried to use postgres with 0.40 * available memory and the performance was awful. Once again, your link explains it all. Basically, between the shared memory-buffer cache and the disk, their is a kernel cache which is as important as the shared memory-buffer, i.e. it needs memory. I am currently using 0.25 * available memory for shared buffer-memory. That leaves about 0.70 * available memory to the kernel cache. Thanks again!
  From Frank Heikens

Data Center Design and Preferences

When either selecting a data center as a co-location facility or designing a new one from scratch, what would your ideal specification be?

Fundamentally, diversified power sources, multiple ISPs, redundant generators, UPS, cooling, and physical security are all desireable.

What are the additional key requirements that someone might not consider on the first pass?

What are the functional details someone might not consider during the initial high level design?

I'd like to approach this from the perspective of designing a large data center or seeking a facility that was designed perfectly from an infrastructure perspective. This question has already been addressed with smaller facilities and workspace considerations here.

From serverfault Warner

• There's good information in the answers to this question. Many of the questions tagged server-room are relevant, too.

  Warner : Thanks, good info. I linked and clarified.
  From Ward

• Location , power costs, water costs/supply (depending on how the facility is being cooled), weather, and natural disasters is what I would add on top of your list.

  From xeon

• Diverse fibre/connectivity routes (including multiple, separate building entry points)

  From James

• I've been lucky enough to have built a few over the years, certainly I'd look to the following points;

  • I'd always go for multiple sites, even if that meant having smaller sites in total.
  • You're right about multiple diversely routed/sourced power supplies and good UPSs, physical security etc.
  • I won't be buying any more AC units for my data centres again, choosing to partially filter the ambient air and use semi-sealed extraction tunnels/pipes/channels to pull the hotter air out - possibly with some form of heat exchange to recoup some of the energy from the heated air. This approach saves a fortune, is 'greener', can support higher watts/rack and much more reliable/available.
  • I'd use solid concrete floor, not raised flooring, this will obviously support higher loads (i.e. fuller racks) with overhead caging carrying mostly OM3 fibre and a few cat5/53/6 coppers for where they're absolutely required.
  • I'd go for fewer, faster, trunked but resilient links to my servers/switches/blades etc. than the old-school waterfall of lower-speed links.
  • With the cost of disk-based CCTV solutions getting cheaper and cheaper I'd cover every row or position in the place and record everything.
  • Every site needs two non-equipment areas - an area in the server room that's fenced off from the racks with a desk and storage for kit and tools, a chair, power, networking etc. - and a second area outside the server room to make calls and get away from the noise.

  I hope this is of help, I might add some more later.

  Warner : Have you discovered a proven solution for cooling alternatives?

  Chopper3 : Yes, but there's nothing to discover as such. 99%+ of IT kit actually PREFERS to work at ambient temperatures yet we keep firing in cold air that costs a lot of money. The problem is the build-up of hot gases from the rear of racks - all that's needed is to remove that. Some of the largest new data centres in the world simply suck out the hot air, you need to ensure you have some degree of sealing around the rack (nothing silly) then you just have fans on the roof instead of AC units. Have a look here; http://www.theregister.co.uk/2008/05/24/switch_switchnap_rob_roy/
  From Chopper3

• I'm sure that the answer will be "it depends." Are you asking for blue-sky, if you had millions of dollars and were trying to run Amazon? Is there a budget you have in mind?

  Ease of expansion is one thing not on your list. If you're renting a cage, how easy is it to add another cage and get the proper wiring between them? If you're building your own, what do you do when you run out of floorspace? Can you destroy offices and expand within the building, can you knock down a wall and make the building bigger?

  Warner : I was writing up an analysis and started thinking about how I would design from scratch, began to wonder if smart people would have ideas that I didn't. One main site has reached physical capacity and we're expanding. Down the road, I'll likely have the opportunity to build another space from scratch using existing space. Right now, I'm picking between data centers. Amazon scope is bigger than I'm thinking but is definitely not outside of consideration, as those ideas filter down.
  From mfinni

• One thing that I don't see already posted is the budget to be able to able to build a very good team of people.

  I recently went cage shopping and found that they pretty much all were peered with multiple tier-1 providers, multiple diesel generators, etc.

  What made me pick the one I did was that everyone there was sharp and dedicated, there were plenty of people on location, the sales managers and projects managers were also great. All the generators and peerings in the world won't help if the guys plug you all into the same generator, or the remote hands don't respond when you really need them to.

  So this may not fall under infrastructure, but in the end it can be more impressive than four vs two redundant generators, 2 vs 3 Internet peers, etc.

  mfinni : Yeah, that doesn't really fall under "design", except maybe accounting for high salaries and good bennies in the operating budget to attract and keep good staff. That's more of a corporate "thing", instead of DC design.
  From Kyle Brandt

Should a sysadmin contractor charge overtime for off-peak hours?

This is not necessarily a server-related question, but more of a system admin question that I think would related to many on SF.

I'm doing Sysadmin/IT consulting for a small company. I only work about 3 days a week for them on average.

If a server goes down or something like that during off hours (nights, weekends, 3am, etc) and they need it fixed during those time periods, should I be charging overtime for that? Or would I not be justified in charging overtime until I've logged 40 hours for the week?

Perhaps calling it overtime isn't the best name. I guess maybe its better to call it an off-peak hourly rate. Anyways I just was curious what other consultants did in these circumstances.

From serverfault Jakobud

• You should communicate your preferences to your customer using the price. If you don't like to do off-peak work, charge extra. Then the customer will evaluate if getting it fixed ASAP is important for them or not.

  Remember to make the pricing clear so that the customer also understands what he's buying and how much it costs. It's good for all parties to have clarity in these matters, the customer knows how much it costs if the server goes down on a weekend, and you get some extra cash for doing things you'd prefer doing in another timeslot.

  Chris W : It's amazing how many emergency problems become less important when the client has to pay extra for them!
  From tstm

• Well you would have had put it in your contract first. I would also say it depends on how expensive to you are to start with. If you pricey, you should probably not charge any more. If you are a pretty good bargain, might be fair to ask for more if after hours work.

  Take into account how often this happens. If they are going down all the time because the developers release into production without testing etc than you probably are justified charging more. If this only happens once and a rare while, the extra relatively few dollars you get is probably not worth aggravating them or fighting over.

  So if you are more expensive, but when stuff goes down you are there to fix it when they need it the most and don't give them nonsense over pocket change billing, you will probably get a better reputation and make more money in the long run.

  Evan Anderson : +1 - Consider the effect on the relationship. My contracts state that I, at my sole option, I may choose to bill labor at a lower-than-required rate for off-hours, emergency, etc, labor. If the situation dictates billing the higher rate, I will. If it feels like the longer-term relationship would benefit from foregoing a short-term gain then I'll bill the lower rate. The Customer's expectation is always set at the higher rate, such that I can pleasantly surprise them with a lower-than-expected bill if the situation dictates.
  From Kyle Brandt

• If you're a contractor it is your job to change as much as you can. In fact it is the role of any businessman to charge whatever you can get away with. Market forces will dictate what that is.

  Let's face it, you let your bank force you to spend hours several times a year interpreting and agreeing (with implicit consent) changes to your terms and conditions. Why? Because they can. Because you can't do anything about it if you want a banking service. It's not fair, it's not just, but it happens.

  So you should treat your customers. Do whatever you can get away with. Of course, often you can't get away with much at all.

  Evan Anderson : -1 - I'd love to be able to meet your Customers. I suspect they'd become my Customers in fairly short order. You've obviously never heard of a mutually beneficial relationship.

  PP : Feel free to study economics any time you have a spare moment. You might even have access to a local library, or something modern called "Wikipedia". You might read about someone called "Smith" - he's not an agent, Mr Anderson, never fear..

  Evan Anderson : I'd rather have a moderately profitable 10 year relationship with a Customer who gives me good references and word-of-mouth advertising than a short term, highly profitable relationship where the Customer leaves feeling like I took advantage of them and never says a good word about me again. I'll be ruthless in trading securities, but not in personal dealings, because I think there's more money to be made with goodwill than with ruthlessness in a personal, high-touch relationship.
  From PP

• Pay scale (including off-hours rate) is definitely something you need to consider when writing your contract. I can't tell you what to do in any particular situation, but here's generally what I've done in the past:

  As a salaried employee I negotiate my salary to cover a reasonable amount of on-call/off-hours time (monthly/quarterly patching, etc.), and when I've been told my salary is "too high" I've politely explained why & that I come at a lower rate if I am guaranteed no off-hours calls.
  If the workload becomes unreasonable or we can't agree on a fair salary then it's time to part ways.

  As a consultant I always have a separate off-ours rate for 7pm-6am, weekends and holidays in order to discourage clients treating me as a "Just call him!" guy. Depending on the client this has been anywhere from 1.1 to 2 times my base rate, and always with a 1 or 2 hour minimum charge attached for waking me up.

  In both cases I make it clear to my employer exactly what constitutes an incident worth calling me for during off hours: Something better be down, and it better be impacting the business (e.g. if something went down but its redundancy partner is up and working it can probably wait).

  From voretaq7

Does servers to >$1000 really have a memory limit of 4GB?

Hi,

I am looking for a server, and when I look at the specs some of the servers can only handle 4GB. Other can handle 8GB or 16GB, and others 64GB.

Can that really be true?

Is this really a hardware limitation, or are they disabling it in the BIOS, so there is no way to use 16GB on a 4GB supported server?

An example is the Dell PowerEdge SC 440. Only 4GB supported, they say.

Would 64bit Linux allow me to use 16GB on a 4GB server?

Sandra

Update:

In case we can trust that Dell haven't written these reviews them self, then one reports that he have put 4x2GB of ram in it.

From serverfault Sandra

• The chipset (and, to a lesser extent processor) capabilities normally dictate maximum memory limitations. I'm sure that some of the limitations are a result of "market segmentation" efforts, but in general the higher-end chipsets that support more memory are more expensive. More memory sockets on the board can also mean a larger board, which translates into lower production line yields and more cost per part.

  Your operating system isn't going to matter re: hardware capabilities.

  I bought a Dell PowerEdge T310 in December 2009 with 8GB of RAM (and a 24GB maximum capacity) for $800.00 w/ a 3 year on-site warranty. There are definitely server-class machines that can support more than 4GB of RAM out ther for sub-$1,000.00.

  Chris S : Some models may also quote the maximum memory the server can handle given currently available options. For instance, if the server only holds two sticks of memory, and the largest memory available for those slots is 2GB, then they may quote the maximum as being 4GB.

  Sandra : That what really puzzled me. It have 4 memory slots, so it is just begging to get 4x4GB =) I just heard that Linux doesn't use the BIOS is initialize the hardware, so I thought if it was something they had disabled in the BIOS, 64bit Linux wouldn't even know of the limit? Thank for the T310 tip.

  TomTom : Well, given the age of the system it is more like 4x1gb that you have now. Like some old servers I have that were bought 3 years ago - looks like same generation. I throw mine out now.

  Evan Anderson : That SC440 very likely won't have a processor that supports 64-bit operation, and certainly won't have virtualization extensions. The machine isn't a boat anchor, but it's not up to any heavy lifting either.

  Sandra : @Evan: I am running 64bit Red Hat on it now with KVM which means it have hardware virtualization.

  Evan Anderson : @Sandra: If you can get the mileage out of the system, certainly push it for all you can. It's definitely within the realm of possibility that it will function with newer memory parts that have a higher density than what was available when it was new. There's no guarantee that you won't have strange issues, though, either. You're venturing into "if it breaks you get to keep both pieces" territory. As long as you're comfortable with supporting yourself go for it.
  From Evan Anderson

• This is a super super cheap low-end server. So it's not surprising that the hardware is limited. Looking online says it's currently worth $200-$400. If you just bought this from someone for over $1000, you probably got cheated.

  Sandra : I bought it when Dell sold them on their site. But at that time I paid ~$1000 for it.

  Broam : How long ago was that?

  TomTom : I bet 2-3 years.

  Evan Anderson : I believe the last SC440 I installed was in October 2006, and I don't think the model was new at that time.

  TomTom : That would match. I have servers of the same timeframe here - AMD 64 dual cores, 3800+ on nforce chips. 4x1gb - that was common for lower end (server) stuff at this timeframe.
  From davr

• There are two different limits that you have to be aware of.

  There is a physical limit that the server will support.

  There is also the limit the operating system will support. 32-bit versions of Windows Server will support up to 4 GB, and 64-bit versions of Windows Server will support at least 8 GB (and much more depending the version of Windows).

  Sandra : Thanks for pointing that out. I am using 64bit Linux.
  From aphoria

• How would linux do that if the hardware can only support 4gb?

  Looks like an oudated system like I have some lying around. THat simple. Modern boards can handle 16gb in end user chips, and 64+ gb per socket server side.

  Sandra : In case it isn't a hardware limit, but they just say it is, so make me pay more so they can unlock it. From what I have heard, Linux doesn't use the BIOS to initialize hardware, so maybe 64bit Linux would allow me to use 16GB?

  TomTom : No, sorry, it IS the hardware. Bascally the chip does not support more. It is a dated product. You don't find the S440 in the current server list at dell. You get them way blow 1000 USD. Nothing with "pay more so they can unlock". More like "pay more so you get more powerfull hardware".

  Sandra : Do you have the serial number for the chip? I would really like to see the specs . It most be the worst chipset ever. =)

  TomTom : Hardly. At that times it was normal - especially for a lower end server. You buy outdated low end equipment, shut up if it is not no par with new high end one. In my case I paid about 2000 USD new. AMD 64 x2 dual core 3800+ on an nvidia nforce chip - low end board, I got 5, still use 3 servers. They can not handle more ram, as it was normal. Worst chip ever? How arrogant is this. Chips and computers evolve. 3 years ago that was a decent low end server. Today... it is not. But 3 years are 1-2 generations.

  TomTom : For my new server I have 64gb RAM, 2x4 cores Opterons and space for 24 hard disc. Bad news - it was also more expensive. You bought cheap, now be a man and live with what you bought.
  From TomTom

NetInstall working on some systems, not working on others

Hi,

I'm having an issue where my NetInstall setup works on some computers and fails on others. I am not able to diagnose the issue.

I created an image of a Mac Mini and then created a NetRestore image using the System Image Utility found on Snow Leopard Server. NetBoot and NFS all seem to be working fine on the server, which is an XServe.

Then I select the NetInstall image from the Startup Disk on a machine. On some of the machines, the process works as expected. On some of them, I see the globe icon blink a few times and then the system boots to the regular hard drive.

I have captured the tracedump and the system.log logs from the server on both cases where NetInstall seems to work and fail. Here is the link that has all the logs

http://gist.github.com/232232

The gist of the failure seems to be from the lack of BSDP DISCOVER in the failure but I'm not able to identify why that exactly is happening.

I'd really appreciate any help on this issue.

From serverfault cduruk

• Hi,

  First off, try a couple of diagnostic steps.

  If you hold down Option at boot rather than the 'N' key you should get a choice of boot devices that includes all your Netboot volumes. Try doing this rather than an 'N' boot and see how that goes.

  You might also try booting one of the 'broken' machines all the way to the desktop and then see if your Netboot image appears as a choice in the 'Startup Disk' in System Preferences.

  I'm assuming that both the 'broken' machines and working machines are in the same network segment so you are sure there are no firewall issues.

  // Tony

  From $pageUsers[$post.author].name

How can I run a shell script upon system shutdown in a Knoppix Live system?

I want to run a shell script upon system shutdown in Knoppix Live (which runs from a writable USB flash drive) so that I can backup some data and ftp it to a remote server. The script works fine but I'm not sure where to put it so that it executes when the system goes into shutdown.

From serverfault GeneQ

• You could save the file as backup.sh in your home directory within the area being backed up.

  Call the script from the .bash_logout file in your home directory. So when the machine goes down, you get logged out, the script runs.

  GeneQ : @Richard Holloway, Tried that. Doesn't seem to work on Knoppix Live. Thanks anyway.

  Richard Holloway : Is /etc/ persistent. I mean can you make changes there which will be saved between restarts? Originally I posted about init scripts and run levels but removed it all from my answer when I read it was Knoppix. Now I see it is on a USB device so may be writable. I am happy to continue with this to get you a workable solution.
  From Richard Holloway

• Suggestion 1

  You could add your script to /etc/init.d and then link it to the appropriate runlevel as a K??my_backup. You will need your script to execute before the network interface is taken offline. On my Debian system, it looks like /etc/rc1.d might be the correct runlevel to link into.

  My concern with this approach is if your script takes a long time to execute. eg FTP site is slow or not available. I'm not sure if the shutdown process will wait for your script to finish, or if your backup script will be killed because it is taking too long. I'll leave that as an experiment for you! :-)

  Suggestion 2

  You could write a little wrapper script to shutdown your system. Something along the lines of …

  #!/bin/bash
  
  /path/to/backup/script/backup_to_ftp
  shutdown -h now
  

  which you could then execute using sudo …

  $ sudo /path/to/script/backup_then_shutdown
  

  Richard Holloway : The script will need to be rewritten to work as an init script. See /etc/init.d/halt for an example. Also you need this in run levels 0 and 6 (ie softlinks in /etc/rc6.d and /etc/rc0.d ) and not in /etc/rc1.d which is single user mode and will probably never get used in this context. Also it will need to be called as an S script not a K script. Best example is to look how /etc/init.d/halt is run. Suggestion 2 will work but he already knows how to run it manually

  GeneQ : @Convict Thanks. I'll give it a shot and see whether it works.
  From Convict

"TCP Sweep" - What is it? How am I causing it?

Hi there,

I've just had an email from my hosting company telling me I'm in violation of their Acceptable Use Policy.

They forwarded me an email from another company complaining about something to do with a "TCP sweep of port 22". They included a snippet from their logs,

20:29:43  <MY_SERVER_IP>  0.0.0.0        [TCP-SWEEP]
(total=325,dp=22,min=212.1.191.0,max=212.1.191.255,Mar21-20:26:34,Mar21-20:26:34)
(USI-amsxaid01)

Now, my server knowledge is limited at best, and I've absolutely no idea what this is or what could be causing it.

Any help would be greatly appreciated!

Thank you

From serverfault Stephen Melrose

• It sounds like they're saying that your machine is scanning TCP port 22 on other machines. If you didn't configure your server to do this then someone else did. Your machine has probably been compromised and malicious third-party software has been installed.

  If that's the case, it's time to level the machine, reload the OS, and restore data from a good backup. You should also do some analysis to determine the root cause of the compromise and prevent it from happening in the future (i.e. immediately after you restore it when another zombie comes along and compromises it again).

  As a practical matter, if you don't know how to do these things then you really need to retain the services of someone who does. Getting your server configured with the least amount of software installed, configured in the most secure manner possible (least privilege, no default passwords, unnecessary features / functions disabled, etc), and on a regular security patch installation schedule will do wonders to prevent this kind of thing from happening again.

  Stephen Melrose : You were spot in. Someone got into my box at around midnight GMT and put some software in /var/tmp. All the files belonged to root except for 1 which belonged to an FTP user. I've removed said software and vsftpd and I'll monitor the situation. Cheers for the help.

  duffbeer703 : Be careful. Many hacking toolkits replace critical system utilities and have a means to hide some of the hacked functionality. If you can nuke the server, unless you have a way to verify the integrity of everything on it.

  Evan Anderson : @Stephen Melrose: Duffbeer703 is absolutely right. The only resolution you should be alright is one that insures integrity of the binaries on the machine. If you don't have a way to boot a freestanding OS and verify signatures on all the binaries on the machine you're really better off leveling and reloading it.

  Stephen Melrose : I'm not sure it was that complicated as it was pretty obvious to find, e.g. `ps aux` showed loads of `./check ip.conf`. However, I agree it's better to err on the side of caution. I'm running a VPS in a cloud environment so reloading is a breeze. I'll do it when I have some spare time. Cheers.

  TomTom : The big problem is - you still dont know HOW they got in. This, interesting enough, is a lot more important than cleaning up the server. Because they will be back.

  MarkR : Sounds like the box needs to be pulled out immediately. Restore your backups carefully.
  From Evan Anderson

• Get a professional to review your server. You likely have to reinstall it - because you got a root kit or something. Normally a server does not sweep. I would take everything I trust offand kill the server with a new install image - faster than checking.

  Then hire some admin to administrate your server. I am sure your hosting provider has managed hosting ovvers where they handle the administration.

  Your statement runs along the line of "The police captured me because I am driving without drivnig licence and basically I have no clue how to drive, what should I do". Running a server on the internet is not exactly trivial and "no idea what this is" is not going to help you here.

  You basically are better off with a managed server (if your hosting provider offers that) than one where you can hang yourself.

  From TomTom