Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Automated Server Benchmarking
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Automated Server Benchmarking

serverbearserverbear Member
edited July 2012 in General

Hey guys, I'm new here. Been a lurker for quite a while, from Melbourne in Australia :)

We've been working (in our spare time) on something for the last few months to help people compare servers in general by performance metrics. I've seen how important benchmarking LEB's are when offers are posted & would love some feedback. I'm really sick of seeing review sites or comparison websites that only show affiliate offers & provide no real value.

Our script does the following:

Grabs general system info (CPUs/RAM/HDD info)
dd - Runs 4 variations of dd for comparison
IOping - 4 variations of IOping (Pings, Seek Test, Sequential & Cached IO)
Network Efficiency - Tests download speed from 9 locations
Tracert to Cachefly
Ping to Cachefly
UnixBench

It's all run from a single command which we append to a particular host & plan. If you personally run the benchmark we generate a report like this: serverbear.com/benchmark/2012/07/22/nhvwlr22zS4yz60O. You can share that or do whatever you like with it, the URL is private by default.

We aggregate some of the key metrics up so people can compare them when searching across plans (Here's VPS plans under $7 as an example: http://serverbear.com/compare/lowendbox), you could overlay SSD onto this report or filter by IO/Unixbench score. What other filters would be useful (i.e. Low End SSD Providers by IOPs).

We'd love some feedback, we're keen to make this as comprehensive as possible plus get as much data as possible, we're toying with the idea of a proper score based on all the data (rather than analyzing it individually). We only released the IO/IOping stuff today so we're a bit light on data.

Disclaimer: We do plan on monetising hosts that do have affiliate programs (we have to make a living), but that won't impact our data. It's filtered by the raw metrics, not by who pays us the most commissions.

Cheers,
Stuart @ ServerBear

Comments

  • Hm... so basically you guys put everything we usually do in a review and a test (hardware/product-wise) and put it into that.

    I like it.

    Thanked by 1ynzheng
  • AsadAsad Member
    edited July 2012

    Looks pretty good, going to give it a try. Welcome to LET :)

    Thanked by 1ynzheng
  • You should give credit/acknowledgement to freevps as you've just added a few more tests onto their script ;)

    Also, I can understand doing a traceroute for Cachefly (as it's anycast, it helps to know where you're downloading from), but I don't think a ping to an anycast system provides much relevance on the network performance.

    Other than that, I do like it and I really like the format/layout of the site.

    Thanked by 2djvdorp ynzheng
  • Yep, we currently support Ubuntu, CentOS & Debian. We automatically install all the packages & run everything.

    Would love to know if there's things that are missing or what could be improved (i.e. multiple runs of dd/iops & taking an average), doing tracert's from multiple locations rather than 1. We're talking with GeekBench at the moment to see if we can integrate that plus looking at some of the Phoronix stuff.

    Conscious to not blow out the time to complete by hours though :)

    Thanked by 1ynzheng
  • ihatetonyyihatetonyy Member
    edited July 2012

    Grumble grumble at the "automatically install all the packages" part, but it's not like they're that many anyway.

    In the middle of running a test right now!

    ALSO: Site does not play nice in Opera.

    Thanked by 1ynzheng
  • ihatetonyyihatetonyy Member
    edited July 2012

    Found a bit of a bug: benchmarked a 128MB SecureDragon and somehow it morphed into a Heroku Fugu on the stat page along the way. Interestingly, the email said "Fugu @ SecureDragon"

    ALSO: 256MB VPS6 turned into "Intel Core2 Quad Q8200 @ VPS6" via email turned into "Intel Core2 Quad Q8200 @ Burst.net" via stat page.

    Thanked by 1ynzheng
  • @serverbear wow, you benchmarked an BuyVM KVM from @Francisco and @Aldryic. Impossibru! (stock is a myth)

    Thanked by 1ynzheng
  • @ihatetonyy Yeah sorry about that, a bit of a bug with plans we don't have already added. We've fixed it & will push out tomorrow, Opera is a big of a weird one, we're focusing on FF & Chrome for now :)

    @djvdorp We've had quite a few benchmarks already from existing BuyVM customers, of course we'd love our own boxes too. We're planning on building a parser to let people know when certain hosts are in or out of stock.

    Any other constructive feedback is appreciated, we can iterate new features pretty quick so let me know what you'd find useful!

    Thanked by 1ynzheng
  • @serverbear said: We're planning on building a parser to let people know when certain hosts are in or out of stock.

    Something like doesbuyvmhavestock.com or lowendstock.com ?

    Thanked by 1ynzheng
  • @djvdorp Both those sites look awesome, potentially something similar but at a larger scale across more hosts (it's easy to scale if they all use WHMCS for example), we also want to integrate the idea of there may be data of a similar host with better performance (for the same or lower price).

    Thanked by 2djvdorp ynzheng
  • How many affiliated links did you stick in there? You should disclose which links are affiliated and which ones are not

    Thanked by 1ynzheng
  • @zhuanyi We're at less than 1% affiliated at the moment, we're more concerned with getting benchmark data & getting plans populated. Disclosure is pretty important to me, but at the same time getting more valuable data is a bigger priority than signing up to affiliate programs :)

    Thanked by 1ynzheng
  • @serverbear said: We're at less than 1% affiliated at the moment, we're more concerned with getting benchmark data & getting plans populated. Disclosure is pretty important to me, but at the same time getting more valuable data is a bigger priority than signing up to affiliate programs :)

    Agreed, but for the ones you have affiliated links please disclose rather than just masking the link with a quick HTTP redirect.

    Otherwise, as far as I am concerned, you could be benchmarking based on the affiliated commission rather than the actual performance :)

    Thanked by 1ynzheng
  • @serverbear said: Opera is a big of a weird one

    Very much so, though turns out what I'm whining about is just a Bootstrap bug.

    Search for , 'OTransition' : 'oTransitionEnd' and change it to , 'OTransition' : 'otransitionend' to support Opera12. To support both 11 and 12, I think you can do: , 'OTransition' : 'oTransitionEnd otransitionend'
     This should be found in bootstrap.js (alle the js together) or bootstrap-transition.js
    

    Once I pop in the Bootstrap-2.1.0 JS the tabs work beautifully.

    Thanked by 1ynzheng
  • jarjar Patron Provider, Top Host, Veteran

    I like it. As for affiliate links, I've no problem with a site like this using an affiliate link for every provider. It's a good way to pay the bills without ads, and it doesn't hurt anyone.

    Thanked by 1ynzheng
  • @jarland said: I like it. As for affiliate links, I've no problem with a site like this using an affiliate link for every provider. It's a good way to pay the bills without ads, and it doesn't hurt anyone.

    Because you are the first one and the only one in LET with an affiliated links in the signature haha :)

    No offence intended, but I like your style where you clearly specify it is an affiliated link and if I like you or if I do not mind give you some commission, then I'd be happy to click on it. On the other hand, if I do not want to do so for any reason, at least I know I should not be clicking on that link.

    Earning money is important, earning money the right way, particularly for a review site, is even more important

    Thanked by 1ynzheng
  • Affiliate links should be fine as long as they do not create any bias.

    Expand: On the other hand, if I do not want to do so for any reason, at least I know I should not be clicking on that link.

    I'm curious, whats the reason? You don't lose anything from affiliate links, and you're helping (at no cost to you) a site thats giving you information for free.

    Thanked by 1ynzheng
  • @antiven said: I'm curious, whats the reason? You don't lose anything from affiliate links, and you're helping (at no cost to you) a site thats giving you information for free.

    Quite frankly speaking, in this kind of situation I really doubt if the reviews/benchmarks are based off the performance of the VPS or the affiliated payout. No offense, but I have seen too many "review" sites which claim Hostgator/FatCow/GoDaddy as the "best" hosting providers out there simply because they offer the best commission payouts.

    Thanked by 1ynzheng
  • And just for clarity: I am OK with affiliated links, and I agree no one should work for free for the valuable information they provide. What I am against is not disclosing them and give the readers the option to click on affiliated links or not.

    Thanked by 1ynzheng
  • @zhuanyi said: Quite frankly speaking, in this kind of situation I really doubt if the reviews/benchmarks are based off the performance of the VPS or the affiliated payout.

    @zhuanyi said: What I am against is not disclosing them and give the readers the option to click on affiliated links or not.

    They were upfront from the very first post about affiliate links, and if it's what they believe they need to make money from the free service they're providing, so be it.

    Unless every part of the website's operations were completely and impossibly transparent from start to finish, someone could come along and take issue with their form of funding and claim that it could/does result in a bias in their results or editorial content.

    Their use of affiliate links is also stated at the bottom of their About Us but it could do better at the bottom of each page, though.

    (If one wants to make sure that their results are not being fuckered with, the script also leaves behind an sb-output.log file in the directory you ran it in that has the raw output from each test.)

    Thanked by 1ynzheng
  • @ihatetonyy said: They were upfront from the very first post about affiliate links, and if it's what they believe they need to make money from the free service they're providing, so be it.

    Sorry if I was not being clear, but this is what I hope to see:

    http://providersites/affiliateLinks (Affiliated Links)
    http://providersites (non-affiliated Links)

    In every single link they send to the providers' website rather than
    http://serverbear/itCouldbeAnAffiliateLink

    LOL :)

    Thanked by 1ynzheng
  • 403 forbidden from iam here

  • @liam Thanks liam, appreciate it.

    @zhuanyi We use the link masking for tracking purposes, I may eventually want to show popular plans per day/week/month. It's a core part of the way the site is built. We may integrate a [a] type symbol beside hosts that are affiliated, I wouldn't say it's high on the priority list atm tho.

    @ynzheng You're the second person that's been having that issue, are you using Google DNS by chance?

    Anyone got any feedback on features & would like more detail on certain things/like to see stuff included? Plus from a web hosting POV what could we do to help hosts show off their benchmarks scores a bit better? (API, Widgets).

    Thanked by 1ynzheng
  • safari in IP4 with 3G connect

Sign In or Register to comment.