Hello All,
We are trying to identify the right approach to do Fail-Over and Load Balancing in Linux server (Centos7), which contains Mariadb(mysql) and web-server (nginx) with some services such as Video On Demand, Games, etc. (streaming through Intranet, not Internet)
Please guide, to select the right technology/solution (Ofcource OpenSource) to do so. Its seems complicated for me because of the dynamic data (eg: If a user watching video and fail over happens, he should get seamless streaming without interruption).
I have considered HA-Cluster (Heartbeat) for Failover and HA-Proxy for load balancing, but still not sure, how to handle dynamic data fail over.
Heartbeat with DRBD can be a good option
On Fri, Oct 24, 2014 at 3:16 PM, Siji Sunny sijisunny@gmail.com wrote:
Hello All,
We are trying to identify the right approach to do Fail-Over and Load Balancing in Linux server (Centos7), which contains Mariadb(mysql) and web-server (nginx) with some services such as Video On Demand, Games, etc. (streaming through Intranet, not Internet)
Please guide, to select the right technology/solution (Ofcource OpenSource) to do so. Its seems complicated for me because of the dynamic data (eg: If a user watching video and fail over happens, he should get seamless streaming without interruption).
I have considered HA-Cluster (Heartbeat) for Failover and HA-Proxy for load balancing, but still not sure, how to handle dynamic data fail over.
-- Siji Sunny -- http://mm.ilug-bom.org.in/mailman/listinfo/linuxers
Hi Siji,
For your requirement, you may also consider using pacemaker cluster setup which can host multiple services. --- http://clusterlabs.org/wiki/Pacemaker ---
What is your requirement actually? How are are planning to achieve the requirements?
On Fri, Oct 24, 2014 at 4:59 PM, abhishek gangwar < abhishekkumargangwar@gmail.com> wrote:
Heartbeat with DRBD can be a good option
On Fri, Oct 24, 2014 at 3:16 PM, Siji Sunny sijisunny@gmail.com wrote:
Hello All,
We are trying to identify the right approach to do Fail-Over and Load Balancing in Linux server (Centos7), which contains Mariadb(mysql) and web-server (nginx) with some services such as Video On Demand, Games,
etc.
(streaming through Intranet, not Internet)
Please guide, to select the right technology/solution (Ofcource
OpenSource)
to do so. Its seems complicated for me because of the dynamic data (eg:
If
a user watching video and fail over happens, he should get seamless streaming without interruption).
I have considered HA-Cluster (Heartbeat) for Failover and HA-Proxy for
load
balancing, but still not sure, how to handle dynamic data fail over.
-- Siji Sunny -- http://mm.ilug-bom.org.in/mailman/listinfo/linuxers
-- Thanks and Regards, Abhishek Gangwar CDAC Mumbai -- http://mm.ilug-bom.org.in/mailman/listinfo/linuxers
Hi Siji,
For your requirement, you may also consider using pacemaker cluster setup which can host multiple services.
Thanks Arvind,
I had a quick study about it, seems like its a good choice for my requirement.
From Wiki -
"The CIB uses XML to represent both the cluster’s configuration and current state of all resources in the cluster. The contents of the CIB are automatically kept in sync across the entire cluster and are used by the PEngine to compute the ideal state of the cluster and how it should be achieved. "
Hoping that this will manage the state of Video Streaming at the time of failover.
Have you tried this before with failover video streaming ?
What is your requirement actually? How are are planning to achieve the requirements?
My major concern is to maintain the state of resources at the time of failover (Specially, video streaming through webserver).
Heartbeat with DRBD can be a good option
Thanks Abhishek,
Someone from ILUGC suggested to use Pacemaker. And with reference to their wiki, seems like it serves my purpose.
http://clusterlabs.org/wiki/Pacemaker
"The CIB uses XML to represent both the cluster’s configuration and current state of all resources in the cluster. The contents of the CIB are automatically kept in sync across the entire cluster and are used by the PEngine to compute the ideal state of the cluster and how it should be achieved. "
What you think ?
On Fri, Oct 24, 2014 at 3:16 PM, Siji Sunny sijisunny@gmail.com wrote:
Hello All,
We are trying to identify the right approach to do Fail-Over and Load
Greetings,
Dear Sunny long time no see... Happy Diwali.
On Fri, Oct 24, 2014 at 3:16 PM, Siji Sunny sijisunny@gmail.com wrote:
Please guide, to select the right technology/solution (Ofcource OpenSource) to do so. Its seems complicated for me because of the dynamic data (eg: If a user watching video and fail over happens, he should get seamless streaming without interruption).
Having worked on some of the aforementioned technologies+ putting virtualisation on top of it, it is almost impossible in case the box serving it fails with the current open source offerings.
Only VMware is somewhere near seamless video streaming and last I heard of it is VERY expensive.
You may call me up and I can explain the issues in detail. If you do not have my number, here it is nine, nine three zero six three three eight five two.
Greetings,
Dear Sunny long time no see... Happy Diwali.
Thanks and Wish you the same.
On Fri, Oct 24, 2014 at 3:16 PM, Siji Sunny sijisunny@gmail.com wrote:
Please guide, to select the right technology/solution (Ofcource
OpenSource)
to do so. Its seems complicated for me because of the dynamic data (eg:
If
a user watching video and fail over happens, he should get seamless streaming without interruption).
Having worked on some of the aforementioned technologies+ putting virtualisation on top of it, it is almost impossible in case the box serving it fails with the current open source offerings.
Only VMware is somewhere near seamless video streaming and last I heard of it is VERY expensive.
Am exploring the possibility of using Pacemaker.
You may call me up and I can explain the issues in detail. If you do not have my number, here it is nine, nine three zero six three three eight five two.
-- Regards,
Rajagopal
Hi Siji,
Hoping that this will manage the state of Video Streaming at the time of
failover.
Have you tried this before with failover video streaming ?
I have not tested such a setup before. IMO, Pacemaker cluster suite will help to provide high availability of the hosted services, in terms of monitoring and failing over the services to other nodes as the policy defined. The PEngine basically does this. It provides the logic and a component, that takes the cluster’s current state, decides on the optimal next state and produces an ordered list of actions to achieve that state.
The failover time of service in cluster will depend upon several factors such as type of resources used, time taken by the each resource in the service group to start up in case of failover/failback, network side delays. To minimize the service delay, a proper tuning will be required not only with the cluster (in terms of failover times) but also with the application which is being hosted.
Depending upon the service hosted, there can be a delay upto few seconds (solely dependent on type of application) at the time of failover which would required to be anticipated by testing the setup prior to deployment.
Kind Regards, Arvind.
On Sat, Oct 25, 2014 at 10:58 AM, Siji Sunny sijisunny@gmail.com wrote:
Greetings,
Dear Sunny long time no see... Happy Diwali.
Thanks and Wish you the same.
On Fri, Oct 24, 2014 at 3:16 PM, Siji Sunny sijisunny@gmail.com wrote:
Please guide, to select the right technology/solution (Ofcource
OpenSource)
to do so. Its seems complicated for me because of the dynamic data (eg:
If
a user watching video and fail over happens, he should get seamless streaming without interruption).
Having worked on some of the aforementioned technologies+ putting virtualisation on top of it, it is almost impossible in case the box serving it fails with the current open source offerings.
Only VMware is somewhere near seamless video streaming and last I heard of it is VERY expensive.
Am exploring the possibility of using Pacemaker.
You may call me up and I can explain the issues in detail. If you do not have my number, here it is nine, nine three zero six three three eight five two.
-- Regards,
Rajagopal
-- Siji Sunny -- http://mm.ilug-bom.org.in/mailman/listinfo/linuxers
There are two parts here:
1. Fail over of persistent data (DB, SAN Volume with Video Content etc) 2. Load balancing on stateless web hosts
Issue#2 is very easy to solve, issue#1 is difficult (unless big-budget that makes SAN mirroring feasible across two data centers).
One way to solve these issues is to use AWS / Azure etc. With these:
1. You can use DB and Storage offering provided by Amazon/Google/Azure 1.a : These offering include HA for DB + Storage 1.b: DB is completely managed by provider (scaling / updates / high availability) 1.c: Storage is mounted as a block device and can be formatted as any filesystem (ext3/ext4) 2. For web tier, you can use any Linux AMI and either 2.a: Use load balancer OR 2.b: Setup monitoring to spin up a new VM if primary crashes (or load goes beyond a threshold in web tier VM(s))
With this setup, initial cost in close to Zero and user will not even notice if any component fails over. Say, user is playing a video and Storage fails over. in This scenario:
1. Http server serving video content anyways has to support streaming ( Http 206 for Partial Content, Range, ETag)
2. Player on client side will always ask for video content in small chunks 3. As any tier fails over, next chunk will be served by failed over tier 4. User's data (DB records for authentication/ preferences) will transparently be failed over by provider.
Hope that helps, Shamit
On Fri, Oct 24, 2014 at 3:16 PM, Siji Sunny sijisunny@gmail.com wrote:
Hello All,
We are trying to identify the right approach to do Fail-Over and Load Balancing in Linux server (Centos7), which contains Mariadb(mysql) and web-server (nginx) with some services such as Video On Demand, Games, etc. (streaming through Intranet, not Internet)
Please guide, to select the right technology/solution (Ofcource OpenSource) to do so. Its seems complicated for me because of the dynamic data (eg: If a user watching video and fail over happens, he should get seamless streaming without interruption).
I have considered HA-Cluster (Heartbeat) for Failover and HA-Proxy for load balancing, but still not sure, how to handle dynamic data fail over.
-- Siji Sunny -- http://mm.ilug-bom.org.in/mailman/listinfo/linuxers
There are two parts here:
Thanks Shamit for your detailed explanation.
- Fail over of persistent data (DB, SAN Volume with Video Content etc)
- Load balancing on stateless web hosts
Issue#2 is very easy to solve, issue#1 is difficult (unless big-budget that makes SAN mirroring feasible across two data centers).
One way to solve these issues is to use AWS / Azure etc. With these:
We may not be having Internet connectivity to do so.Everything happening in Intranet.
- You can use DB and Storage offering provided by Amazon/Google/Azure 1.a : These offering include HA for DB + Storage 1.b: DB is completely managed by provider (scaling / updates / high
availability) 1.c: Storage is mounted as a block device and can be formatted as any filesystem (ext3/ext4) 2. For web tier, you can use any Linux AMI and either 2.a: Use load balancer OR 2.b: Setup monitoring to spin up a new VM if primary crashes (or load goes beyond a threshold in web tier VM(s))
With this setup, initial cost in close to Zero and user will not even notice if any component fails over. Say, user is playing a video and Storage fails over. in This scenario:
- Http server serving video content anyways has to support streaming (
Http 206 for Partial Content, Range, ETag)
- Player on client side will always ask for video content in small chunks
- As any tier fails over, next chunk will be served by failed over tier
- User's data (DB records for authentication/ preferences) will
transparently be failed over by provider.
Hope that helps,
Yes..Will definitely explore this further.
Shamit