Creating a CDN ?

SpeedBus

Hello,
I've read a lot on LEB (comments) that people have their own private CDN's running, I wanted to make my own for my image host to speed up the site as 1 static VPS can't handle all the load

If anyone has an how-to or an idea / way to get this done, please do reply

Thanks !

Happy New Year !

breton

http://lmgtfy.com/?q=creating+cdn, duh

drmike

We discussed the idea a bit here:

http://v2.lowendtalk.com/questions/10367/create-your-own-cdn-with-vpses

edit: I have to admit that I'm still torn on the idea. Every time I hit a site that I know is running a CDN, things seem slower, not faster, to me.

joesdc

I have been looking at Onapp for a cloud hosting platform and I know they have a CDN product as well. you might check them out. I have had really good discussions with them over their products. I have not personally looked at how their CDN product works but just came to mind when I saw your post.

SpeedBus

@breton : I did see the Unixy link, look's a bit complicated

@drmike : I'm planing to use the CDN just to distribute the load and bandwidth usage, nothing else, speed does not matter

@joesdc : lol, The price of OpApp CDN would burn holes in my pockets and my pants aswell.

SpeedBus

I'll just use round robin DNS Does the work

drmike

@SpeedBus said: I'm planing to use the CDN just to distribute the load and bandwidth usage, nothing else, speed does not matter

What software are you using for a image host? I know with some of the social networking software we use that allow uploads, each user has their own upload directory. Our plans for those, if any of those sites ever got really big, was to map those directories in some way to other servers elsewhere.

@SpeedBus said: I did see the Unixy link, look's a bit complicated

They have a service as well. Mentioned in one of the updates at the top of their page.

SpeedBus

@drmike : Nah, mine's quiet simple and light :P http://imgair.net

I'll have a look at their prices, if it's below $35/month I'll think about it anything above that I'll get a dedi :P

drmike

@SpeedBus, this is the one you coded, right?

Leapswitch doesn't have a solution? I ask since they talk a couple times about clouds on their site.

Centaur

I think it might be possible to just use subdomains, with another VPS. For example. When a link is generated for a static file, then you can use simple php to change the subdomain to point to whatever "CDN" you have. Like:
static0.thedomain.com for the static files on your home server and then
static1.thedomain.com for the static files on the other VPS.
Then randomly switching between 0 and 1, will reduce the load. If the directory is different, you can just change it in the subdomain or you can use an if statement to switch between the links. This way it will put some of the load on your other server.

The problem will be the synching. However, it is possible with create fall back links, if the one isn't available. But for that you will have to use javascript (just like they do with jquery CDN hosted stuff http://css-tricks.com/snippets/jquery/fallback-for-cdn-hosted-jquery/). Or you can upload the file to both servers when it is created. Another option is server-side checking, but this might create an extra lag.

SpeedBus

@drmike : It's partly coded by me and a few other people, you can check 'em out in the about page, Well, I haven't spoken to them about their Cloud offering, I could ask them later on, As I wanted to create the CDN myself and run it just to serve the static files (Logo, Wallpaper and other icons on the site)

@Centaur : Hmm.. There won't be much content to sync, just 5 static files (images), I'll give round-robin DNS a try

drmike

@SpeedBus said: just to serve the static files

Well if that's all you wanted to serve, I'd just get another VPS (or even shared hosting) elsewhere and do it as a subdomain. I know we do this on one of the forums I help moderate. The forum is on a dedicated box and the static files are on a shared hosting site in another datacenter. The hosting account is just as @Centaur suggests, a subdomain.

I think most of us are thinking that you're wanting to spread out the uploaded image files....

SpeedBus

@drmike : Hmm.. I'm already doing that, i.imgair.net serve's the static content now.. wanted to add one more VPS to it

Centaur

How big are these static files that you want to use a CDN for?

titanicsaled

The other option is to write a php script to compare the load on all of your servers and provide a download link on that server.

You can also do the same using users location based on ip addresses.

Adam

@SpeedBus said: Hmm.. There won't be much content to sync, just 5 static files (images), I'll give round-robin DNS a try

If it's only 5 files, I would do what Centaur said, except change it up a little:

• Use mod_geoip (HttpGeoIPModule for nginx)
• Cron based CURL/WGET to check if server is up, and the load (Request a php page that outputs the current load)
• Save server load to database/file. If empty, then server is down. If not then rotate based on lowest load.
• Save server to session, to eliminate calling script for every load

Below is a simple script that I whipped together that works with NGINX and it's GeoIP module (doesn't do load, just a simple rotator):

<?php
// start session
session_start();

if (!isset($_SESSION['server'])) {
    // determine county code (using country database is quicker)
    $country = getenv('GEOIP_COUNTRY_CODE');

    // define servers and directories
    $s1 = '192.168.1.1/img/';
    $s2 = '127.0.0.1/img/';
    
    // country code : server to use
    $servers = array('US' => $s1,'CA' => $s1,'MX' => $s1,'GB' => $s2,'FR' => $s2,'DE' => $s2);
    
    // check IP against array
    $server = '';
    foreach ($servers as $key => $val) { if ($key == $country) { $server = $val; break; } }
    
    // if server value is empty, random a server
    if (empty($server)) { $server = $servers[array_rand($servers)]; }
    
    // set session to avoid duplicate calls
    $_SESSION['server'] = $server;
}
?>

In your html, simply put the following before the filename:
<?php echo $_SESSION['server']; ?>

SpeedBus

@Centaur : 10kb - 500kb max

@titanicsaled : The VPS's are not loaded much, they just run Cherokee or Nginx or Lighttpd, nothing else

@Adam : Cool, looks neat, I'll give it a try Is there a way to just keep rotating them, irrespective of the location of the visitor ?

Adam

<?php
// start session
session_start();

if (!isset($_SESSION['server'])) {
    // define servers and directories
    $servers = array();
    $servers[] = '192.168.1.1/img/';
    $servers[] = '127.0.0.1/img/';
    $servers[] = '192.168.1.2/img/';
    
    // random a server : set session
    $_SESSION['server'] = $servers[array_rand($servers)];
}
?>

In your html, simply put the following before the filename:
<?php echo $_SESSION['server']; ?>

SpeedBus

@Adam : Thanks

One last thing,
If I want to load up, logo.png then would this work,

<?php echo $_SESSION['server']; ?> img src="logo.png" 

I removed the < > tags from img src, as it wouldn't display :P

Adam

<img src="<?php echo $_SESSION['server']; ?>logo.png">

SpeedBus

@Adam : Thanks

Adam

Here's some extra code for you

This will simply add a fail-safe in case one of the servers is down.

<?php
// start session
session_start();

if (!isset($_SESSION['server'])) {
    // define servers and directories
    $servers = array();
    $servers[] = '192.168.1.1/img/';
    $servers[] = '127.0.0.1/img/';
    $servers[] = '192.168.1.2/img/';
    
    // random servers : set session
    $array = array_rand($servers,2);
    $_SESSION['server'] = $servers[$array[0]];
    $_SESSION['fail'] = $servers[$array[1]];
}
?>

For your img's, use this:
<img src="<?php echo $_SESSION['server']; ?>logo.png" onerror='this.onerror = null; this.src="<?php echo $_SESSION['fail']; ?>logo.png"' />

SpeedBus

@Adam : Thanks !!! I'll setup the VPS's and test this outt

steven123

lovely, i like the idea

dnwk

@SpeedBus said:
Hello,
I've read a lot on LEB (comments) that people have their own private CDN's running, I wanted to make my own for my image host to speed up the site as 1 static VPS can't handle all the load

If anyone has an how-to or an idea / way to get this done, please do reply

Thanks !

Happy New Year !

I have about 4-5 LEBs around the world and sitting there idling. If anyone would want to dump something on it. Let me know. But I would not expect any traffic more than 500G/month

Mun

@SpeedBus said:
I'll just use round robin DNS Does the work

You are going to dislike this, as if one server goes down and you have two servers, then 50% of your viewers will not get the content.

Mun

gbshouse

You can try to use GeoDNS

Holoshed

Now this was an interesting necro post. In this day in age if you want to run your own network of content servers around the world I would agree with @gbshouse and go with GeoDNS. Unless you run a site with a lot of static files I do not see a benefit to a CDN.

dnwk

@Holoshed said:
Now this was an interesting necro post. In this day in age if you want to run your own network of content servers around the world I would agree with gbshouse and go with GeoDNS. Unless you run a site with a lot of static files I do not see a benefit to a CDN.

And there are lots of LEB's around the world which you can take advantage of to build a Low end CDN.

ihatetonyy

@Holoshed said:
Unless you run a site with a lot of static files I do not see a benefit to a CDN.

Streaming audio/video. The same benefits as a static media CDN apply -- faster speeds, lower latency. One other key benefit from running (high-traffic, anyway) streaming on one VPS is that your provider won't strangle you for maxing out a node's port speed.

VPSCorner

Considering the price of third party CDN this is a very good idea. Something I might actually look into.