Reference:

http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html

Assumptions:

1. Server1 IP: 10.1.1.1
2. Server2 IP: 10.1.1.2
3. We will completely replicate all databases
4. Servers are not in production yet. If they are – be careful when rsyncing data

Notes:

• This setup can be used in HA environments, provided that only one of the servers is used for writes (INSERTs, UPDATEs, DELETEs) at a given moment
• In case one of the servers crashes, and data corruption occurs (by corrupting binlogs or innodb data files), there is the possibility that replication will fail and data will have to be re-synchronized from the known good server to the other one

Read more…

Without strict standards, the internet would be just a bunch of interconnected networks, and finding valid information about some specific resource would be next to impossible.

The most simple example (not directly related to the Internet or IT, but illustrating the main idea):

Imagine the modern telephone system. Each country has it’s own area code, each state/district/region has it’s own area code too, inside the country. Each smaller region has an area code too.

For example, you want to call someone in Vienna, Virginia, with the local number 123 4567.

You will have to lookup the U.S international code first (+1 in this case). After, you will need to find the Virginia’s area code. You will use something like a phone-book for this. In the phone book – we can notice that Virginia, U.S has multiple area codes (276, 434, 540, 571, 703, 757, 804 in this particular example). In the same phone book, we find that Vienna’s area code is 703, so, we will have to dial: +1 703 123 4567 (where 123 4567 is the local phone number).

So, to find the intermediary codes you have to dial in order to be able to reach the local phone number, you have consulted a phone book, or the equivalent of a “Directory Service” (The information in the phone book being the directory itself).

A similar logic is hidden, for example, behind the DNS (Domain Name System) hierarchical naming system. We won’t go to deep into DNS structure, but in order to find valid information about a specific domain name, you will have to take similar steps as with the phone system.

Read more…

What to account for when starting a file hosting platform
*The following article is in no way a complete “HOWTO” on designing and implementing a file hosting platform. It is to be taken as a series of recommendations on the subject*

First of all, we need fast storage servers.

One would think that a few bulky servers with lots of HDD space, fast RAID10 arrays and Gig uplinks would be enough. In a real-world situation – this is not too correct. You will soon notice that you can’t get even 2-300Mbit traffic from these servers, since the bottleneck in this situation would be the hard drives, which will have to do random multiple reads and writes concurrently.

A better option would be to use a few raid1 arrays. or even better, a few separate drives, since MTBF is pretty high for the current HDD models on the market.

This way, for the storage servers, you could use Dual or Quad CPU machines, with 10 – 20 x 1TB hard drives (how many drives – depends on the SCSI/SAS/SATA controller model), also the system partitions should be placed better on a separate RAID1 array. Usually – 2×36 or 2×72 GB SAS/SCSI drives would be a better setup.
This will ensure necessary performance at a reasonable cost.

Read more…

People want their servers, and of course the projects that are running on them, to be available all the time, and to bring maximum revenue at the same time.
In a real-life situation, it is more than clear that pure 100% availability is impossible to achieve, especially if a single hardware device is involved. There can be unpredicted factors that affect availability, either at software or hardware level.
Most common examples:

• power failures
• network device failures
• sysadmin’s mistakes

Most of these factors can be excluded, or their impact can be reduced to a minimum by a careful initial design and planning, and eliminating any SPOF (*single point of failure), whenever possible of course.

We also have to define the notion of downtime: it is the amount of time when the service is unavailable, or the system fails to provide the services it should be providing.

Downtime can be: Planned or Unplanned
In a HA (*high-availability) environment, we need to exclude unplanned downtimes.
Planned downtime – is the result of maintenance procedures executed on the system. This may include:

• hardware components replacement
• applying security patches or OS updates, that require a system reboot
• performing hardware upgrades
• system redesign

In this article we are designing a high-availability solution for a web service, using open-source software.

Read more…

This article is covering the configuration of the DSR load balancing mode  on the Alteon load balancers running WebOS 10 and a couple of Linux systems acting  as real servers.

Alteon 180/184 series load balancers is a cheap solution to handle at least 1 Gigabit of load balanced web traffic, when using DSR.

As per Wikipedia, load balancing is a technique to spread work between two or more computers, network links, CPUs, hard drives, or other resources, in order to get optimal resource utilization, maximize throughput, and minimize response time. Using multiple components with load balancing, instead of a single component, may increase reliability through redundancy.




DSR is a way for outbound traffic to bypass the load balancer, sending traffic directly to the default router of that network.
DSR uses the loopback interface on a server to spoof the address of the VIP (virtual ip address) on the load balancer when sending traffic out, making it look as the load balancer sent the packet instead of the server, thus eliminating the need for the load balancer to process that traffic. The loopback interface is a special kind of network interface inside the machine. Usually, it is used only by the operating system for internal network communications, but it can be used for other purposes, such as DSR.

Basically there are four necessary steps for DSR setup:








1. Configure the IP alias on the server loopback interface with the IP address of the load balancer VIP.



On a RedHat/CentOS -like server the config file /etc/sysconfig/network-scripts/ifcfg-lo:0 will look like this:

DEVICE=lo:0
IPADDR=10.0.0.1
NETMASK=255.255.255.255
ONBOOT=yes




On Linux running kernel 2.6  there is one more thing to adjust,  because the Linux boxes will respond to the ARP requests for that VIP, when they are not supposed to. This can be prevented by specific kernel arp settings, added to the configuration file /etc/sysctl.conf  (2.6 kernel only) and rebooting the server:


net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.eth0.arp_ignore=1
net.ipv4.conf.eth1.arp_ignore=1
net.ipv4.conf.all.arp_announce=2
net.ipv4.conf.eth0.arp_announce=2
net.ipv4.conf.eth1.arp_announce=2




The following commands may be used to change the settings interactively during runtime:


echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/eth1/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/eth1/arp_announce




Unfortunately there seems to be no general and simple solution for for kernel 2.4. For additional information about the nature of the problem (and other solutions) check http://linux-ip.net/html/ether-arp.html#ether-arp-flux .



2. Configure the web server to bind to both the real IP address (so the load balancer can still perform health checks) and the new loopback IP address.



A config snippet example for Apache  :


<VirtualHost 10.0.0.1 10.0.0.100>
ServerAdmin support@example.com
DocumentRoot /var/www/html
ServerName www1.example.com
ErrorLog logs/error_log
CustomLog logs/access_log common
</VirtualHost>


3. Point the default route on the web servers directly towards the router (rather than through the load balancer).



In this example, run on the Linux server :



route add default gw 10.0.0.254



and save this configuration in the init scripts (/etc/sysconfig/network OR /etc/sysconfig/network-scripts/ifcfg-ethX) .

4. Configure the load balancer to enable DSR.

In this test scenario I`m using the following ip addresses :


10.0.0.1 and 10.0.0.2 as web servers RIPs
10.0.0.100 the Alteon VIP
10.0.0.254 the router, acting as gateway for servers as well as for the Alteon.

A sample config of the alteon load balancer with two real servers behind a single VIP address.




The most important advantage of using DSR is performance, because the load balancer handles about one packet in for every eight packets out, depending on the traffic profile, the load balancer does substantially less work.
The disadvantages of using DSR is a most complex setup and because this method may perform only layer 4 load balancing, as layer 7 (URL parsing and cookie persistence)  requires the ability to completely proxy a connection.



References:

1. http://en.wikipedia.org/wiki/Load_balancer
2. http://lbwiki.com/index.php/DSR
3. http://www.inlab.de/balanceng/faq.html
4. Tony Bourke: Server Load Balancing, O’Reilly, ISBN 0-596-00050-2

RSMail is a complete mail server management software.
People can easily manage their mail server, POP3/IMAP server, configure spam filters and antivirus protection in its web based interface.
We created a small page with several screenshots, for people interested in its actual state.

One of our customers asked to made a script which will suspend his hosting customers if they are using too much resources.

The goal was to collect cpu usage by

1. apache and mod_php
2. mysql
3. cgi/daemons
4. mail

The tool will get actual resource usage every 5 minutes and save the data in a dbm database.
Actually we started with apache resources usage.
The utility is collecting cpu usage per user , vhost and even requested url.
The first try could be downloaded here, available for Cpanel servers only.
I do not recommend at all to run it on non-Cpanel servers.