Into the clouds

Posted by Giles Cambray on 18 April 2008

There have been a couple of big announcements this week that probably aren't getting the attention they deserve, especially when you consider the paradigm shift that they actually represent.

Last weekend Google announced AppEngine, a mixture of a full-stack web development framework and hosting services to run AppEngine apps on. Although initially only supporting the (so hot right now!) Python language, Google have stated their intention to include other languages in the future.

This was followed by Amazon, who announced two significant additions to their EC2 elastic cloud computing service. Firstly they revealed the soon to be released feature of persistent storage of their EC2 service. Secondly, they announced professional support services to go alongside the EC2 service. This comes after the recent news that they now allow for static IP addresses to be assigned to their EC2 instances.

For those of you that read the last two paragraphs and now have absolutely no idea what I am waffling on about (well done for making it this far!), it can be boiled down to the three stages of hosting web sites.

Stage 1 (circa the mid to late 1990s) was real DIY stuff. If you wanted to put a web server online it generally meant selecting the hardware, procuring it, installing whatever operating system and application servers onto the box, driving it to the data centre, racking it, plugging it all in and then praying that the hard disks didn't explode in 6 months time. Life was painful.

During stage 2 (to the present day) things became more service oriented. If you wanted a server you gave a hosting company a call, told them what you wanted and a couple of days later they gave you the access details of the server. If the hard disk exploded you phoned the hosting company and they put a new one in straight away. Life was better.

In stage 3 (from this week onwards) things are getting really smart. The key difference between stage 2 and 3 is simply this: The server in stage 2 exists physically somewhere in a data center. You could locate it in the data center, pull the power cables out and your website would stop responding. In stage 3, you would be very hard pressed to even be able to find the server that your website is running on. In fact, you would be hard pressed to find out what country the server is located in. This is all made possible through technology known as virtualisation. If the hard disk fails, your website still responds. If there is a problem discovered in the server that is running your site, it hands over control of the site to another server and alerts a technician that it is unwell.

Think of stage 1 as owning a car before car manufacturers thought of selling after sales service, stage 2 as renting a car that is serviced by roadside assistance, and stage 3 as taking a cab.

Now think about the cost implications of all 3 stages. Buying a car, renting a car and taking a cab. Now think about the speed that you can alter your business offering depending on these three stages.

Welcome to cloud computing.

Notes on the iPhone

Posted by Ben Rometsch on 18 March 2008

Having struggled with Windows Mobile Devices for a couple of years (they were good but crashed ALL the time, like when you had an incoming call) and a Nokia E61 (solid but oh so boring) I finally took the plunge and bought an iPhone.

I didn't have an iPod, so the cost factor was offset to a certain degree on that account, and fortunately O2 tripled the number of free minutes I get a couple of months ago, without me asking, which was nice. That puts the monthly tariff pretty close to what I was paying T Mobile with a similar package, and the O2 network is much, much better than T Mobile, but I digress!

Cutting through all the"Jesus Phone" hoo-haa, I have come to the conclusion that it is, indeed, a magical device. It's magical for three reasons.

It doesn't have GPS, but it knows where you are...

A recent free software upgrade by Apple provided a location based service on top of google maps. Now, the phone doesn't have a GPS chip, and it figures out where you are based on a combination of cell tower triangulation and Wifi hotspot database cleverness, but the bottom line is, I can be anywhere in London, hit a button and have the device show me, to within about 10 yards, exactly where I am.

At first I thought that it was just lucky at guessing where I was, but as I have come to use it more often, I can safely say that it is actually *better* than GPS. Why? Because I don't need to have a clear line of site to the GPS satellites, it doesn't take 5 minutes to get a signal lock and the accuracy is more than "good enough".

The other day I got a bus that I don't normally use, and wasn't sure which stop to get off at. I had the destination post-code entered into the Google Maps application, so the phone knew where I wanted to be. Every couple of minutes (and this was sat on the bottom deck of a double decker, try that with GPS...) I'd hit the "find me" button and it would update my location. No stress, no hassle, and I knew exactly where to get off. Brilliant.

It isn't 3G but is faster than my old 3G phones.

Huh? Really, it's true! 3G networks are a bit creaky, and the technology is still maturing. The network coverage is not great, and if you are on a train your 3g phone will spend most of the journey bouncing between 2G and 3G like a yoyo. Don't try and use it when it is doing that! The bottom line is that, when you need to get hold of some information on a mobile, having 3g isn't much of an advantage. In fact, I think it's a disadvantage.

The Edge/GPRS/Wifi connection stack on the iPhone is really superb. It can handle moving from Wifi to Edge to GPRS without a hiccup. You get the best connection available to you at the time. This, along with the simply superb implementation of the Safari Web Browser and hand gestures mean that it is actually *quicker* to get access to information online with the iPhone than it was with my old Windows Mobile Device (and that was using HSDPA, which is meant to be "faster" than 3G!).

It is an inanimate object, but it makes me smile.

The quality of the interface design is truly astonishing. I've had the phone for months now, and use it a LOT, but I am still finding all these little interface embellishments that make me smile. The windows mobile devices used to make me angry, on a daily basis. The iPhone makes me happy on a daily basis.

The ramifications of my Mum and Dad missing Eastenders

Posted by Ben Rometsch on 22 February 2008

Dave Tomlinson of the ISP PlusNet has added a fascinating post to the company blog discussing the effects of the BBC iPlayer on their network traffic. I talked about this a couple of months ago, and the effects of the iPlayer appear to be taking hold.

The two killer stats, in my opinion:

• 66% growth in volume of streaming traffic since 1st December
• Cost of carrying streaming traffic increased from £17,233 to £51,700 per month

And that's the increase in a single month!

At present, it's technically quite tricky to get the iPlayer working on your living room TV. It can be done (Home Theatre PC, connecting your Laptop to the TV etc etc) but it's not easy. Funnily enough probably the simplest way of doing it at the moment is with a Nintendo Wii, making use of the excellent Opera browser and associated Flash plugin. I digress. What I mean, and the relevance of the title of this post, is that for my Mum and Dad it's simply not going to happen on their living room TV set. They will have to sit crouched around their desktop PC.

What they need is something like an Apple TV, but even simpler and cheaper. Something that comes with a wifi connection, a flash player and an HDMI output. You could probably make a slice of profit retailing them at £80. Comes with Linux and a flash player capable of decoding the H264 codec. Plug it in, connect to your wifi point and away you go. There's definitely a nice business there, and you can be sure it will run on Linux!

Anyway, once that comes onto the market, you can expect those figures of Dave's to go absolutely crazy.

I'd say that at this point there are a few things that we can now predict with a fairly high degree of certainty:
1. Any ISP that is living on the edge in terms of profitability will be bust within 2 years
2. A lot less unmetered bandwidth products from service providers
3. The BBC's bandwidth bill is going to start getting seriously, seriously big
4. A lot of consolidation within the ISP market, with the big players that can strike good deals with the top tier networks hoovering up the smaller providers.

And finally, I cant see anyone using the terrible, terrible P2P client software that they are using as one of the delivery mechanisms. It's practically malware, doesn't work nicely with the channel 4 or ITV software, is horrible to install, does nasty things to your machine, it's just terrible. People don't care about the fractional loss in quality. They want instant streaming with no software to install, and that means a flash client.

Anyway, if you are reading this and have made that Linux based flash client, please send me one so that I can placate my mum when she calls me about her extraordinarily large bandwidth bill.

Testing by turk

Posted by Matthew Evans on 21 February 2008

Testing our web applications has always been a difficult area for us at Solid State Group. When we were smaller we used to pass on the QA stage of the web build process to developers that were working on different projects in order to get fresh eyes on the site. Then came the time to hire dedicated testing resource for QA testing and to give good feedback on developer unit testing. Dedicated testing resource makes a difference to the quality of the end product and we are always keen to fnd new efficiencies in this area.

We've been usng selenium for some time now for post build testing and regression testing after adding new functionality to a project. Selenium is a great tool for automating tests that you know should return certain conditions after certain triggers, but it's never been very good at initial release testing. Trying to build the tests for selenium always raised more bugs than actually running the tests.

It seems that there is no replacement for the human eye and the ability to understand the complexities of an application, however with humans comes the overhead of management and transfer of domain knowledge.

Wouldn't it be great if the interface into testing were automated and mechanical but the actual execution were done by humans? Like the mechanical turk for testing! Well theres a new testing sheriff in town and he's called utest. This new service, to be launched in BETA in April does exactly that. It allows web applications to be tested by a community of testers regitered on the utest site. Projects are specified technically and uploaded to the site, and registered testing crews log bugs against your application. Once you verify they are in fact real bugs and not just text pasted from slashdot then the tester gets paid for that bug.

Personally I think it's one of the better uses of the so-called "social networking" revolution and we are applying to be in the BETA. It's going to be very interesting to see how the test scripts are defined and how the interaction between testers and testees works out.

Let's hope it's not just a load of the latter.

Digital piece of mind

Posted by Matthew Evans on 08 January 2008

I'm fairly into technology. I would say I'm a bit of an early adopter, paying more to have the gadgets that aren't quite ready for release yet. I remember having a mobile phone in 1994, it wouldn't fit in my pocket, it cost me the best part of £200 plus a hefty line rental but no-one would call me on it because of the rates. I remember having a digtal camera when they were the size of a house brick, it didn't have a screen on the back and it took pictures at 640x480 pixels. I also remember getting an early windows mobile that used to have about a 40% chance of successfully anserwing an incoming call, would open the web browser during text messaging and would pop up a reminder for a meeting and simultaneously take a photo of the inside of my pocket. Useful gadgets.

The upshot of being an early adopter is that I have a large amount of personal data thats getting larger by the day. I have around 60Gb of music, 20Gb of photos and videos and around 10Gb of personal files, data and other bits and pieces. Now that my son has been born, my photos are more important to me and I have started getting a bit more serious about backing up all this data. Previously I have been writing all this data to disk, then CD, then DVD. More recently I have been running multiple drives in a network and mirroring content until finally going for some internet based storage.

About 6 months ago I came across an interesting combination of tools that allows me to back up all my data automatically onto the net and never have to worry about crashing another drive again. I have subscribed to Amazon S3 (simple storage service). This is basically rental of data storage on Amazons servers. You sign up, and they charge you for uploading data, storing data and downloading data. The prices are actually very reasonable, I pay about $1.5 a month at the moment.

To use this service I have downloaded and installed a great little free app that maps this storage as a drive in windows. It's called jungle disk. Now I have a spare virtual hard disk drive with limitless storage.

Now to finish off I downloaded and installed a shareware automated backup application called syncback from 2BrightSparks. This program simply allows me to specify a bunch of folders and files to copy from one drive to another. You can set it to run whenever you want and it has all sorts of cool features like sync/copy, error reports and more.

So now I don't have to worry about my data being lost anymore, syncback runs nightly and my data is copied through jungle disk to Amazon S3, and I can get to my data from any PC with jungle disk installed. Digital piece of mind has been achieved.

On a similar subject in the office, we use a network access storage (nas) box to store all our valuable data. This neat little box of disks from Thecus can handle up to 2Tb of data over 5 disks in raid or a bunch of other configurations. the cool thing is it's all networked access to I can now get to all the office files from home or anywhere, even from my phone.

Google launches data (and lore)

Posted by Matthew Evans on 06 November 2007

So Google have launched their "Android" operating system for mobiles.

Check out the launch article from the BBC

It seems like the main focus about this release is the fact that this new operating system for mobiles will be open source and will not require a license, making it quicker and easier for software firms to develop applications.

Google and friends are also claiming the new phones will make the internet experience "better than on a PC". Quite how thats going to work with most phones only having a screen the size of a matchbox is, at ths moment, unclear.

What is clear (ish) is that Linux is about to become one of the biggest contenders for mobile operating systems over the next few years, and with the open source community able to get fully involved, and get full access to all the devices on the phone, things in the mobile sector might actually become more stable!

I recently got a mobile that has a built in GPS unit, once that becomes available as a device within Linux and with the onset of mobile mashups, I fully expect some very cool location based applications to be available in 2008.

As a content management company, whats clear to us is that the system that manages content in your business environment better be able to break it down to asset level, push it through different templates and channels, spew it out and through XML and other trendy standards and generally be as flexible as possible. If you are going to be loading content through a mobile connection you want to be able to get at the individual assets, not have to parse hefty XHTML.

Roll on the tuxPhone.

Glassfish is swimming in our office

Posted by Ben Rometsch on 02 October 2007

Ever since we started Solid State Group, our Application Server of choice has been Tomcat. 5 and a half years ago it was a little rough around the edges but it had that killer feature that all startup companies love - it was completely free!

In the last five years we've gone from version 3 (which was kind of stinky) to version 5.5 which smells of barbequed Vietnamese skewered pork. It is not, however, without its issues that are fine if you know what they are and the best way to deal with them.

Matt Raible has an excellent blog that is full of useful stuff. I noticed a post on deployment and startup times for Glassfish in comparison to Tomcat. Glassfish is the shiny new application server from Sun. Not only is it completely free, but it has a pile of new stuff that we don’t use. If we aren't going to use the new stuff, why all the interest, I hear you ask? Well, if it can cut down on our maintenance costs I'm all for it.

When we started the company we had a single server in a data center; maintenance wasn’t such a big issue. Now we have a whole pile of servers that power sites for a large number of our clients. Although we partner with a great company to look after our physical infrastructure, we've always believed that our clients would get the best level of service if we looked after the software part of the machines ourselves. No passing the buck when things go wrong.

Anyway, I'd been waiting to have a play around with Glassfish when I had a bit of downtime, and I just got around to installing it and trying to deploy a recent WebDeck build with one of our current project builds. What happened was testament to the standards that Sun have been laying down ever since Java was born.

It worked. Perfectly.

Now it's time to give it a proper round of load testing and see if I can break the thing. If I can't, it might be time for Tomcat to move over for the Glassfish!

I want my MTV...and Youtube and iPlayer and...

Posted by Ben Rometsch on 14 August 2007

I want my MTV...and Youtube and iPlayer and...

There are interesting rumblings afoot in the world of online video, Internet Service Providers and TV broadcasters. There are a number of not-that-interesting-to-most-people discussions that spin off from all of this, but the main question really is: who is responsible for paying for the distribution of online video.

Here's the current situation. At present, most people watch TV in their lounge, in front of their big TV from a signal sent either by Sky Satellite, Virgin Media Cable or over the air in the form of Freeview Digital or the old-school analogue signal. They may well use some sort of PVR or VCR to time shift the viewing, but on the whole the actual content they watch was broadcast out to millions of people from one single transmission.

That's fine if everyone wants to watch the same thing at roughly the same time. For those that want to watch whatever they want, whenever they want (and to a certain extent wherever they want) single transmissions are useless. They tie you to roughly 1 spot at roughly 1 time. Kids who are growing up not knowing a world without the internet and on-demand video will find this sort of limitation absurd, in the same way their parents could not consider a world without television, in the same way their parents could not consider a world without radio. And so it goes.

The venerable institution known as the BBC lurched into the 21st Century recently with the launch of it's iPlayer service. It allows you to download and watch recently broadcast video from a PC. There are a number of pretty horrible restrictions (you need Windows XP, Internet Explorer, videos expire after 30 days, you cant watch stuff more than 7 days old, you cant copy videos to another device) which they have rightly received a lot of flack for.

One thing that has not been prominently noted in the press is the fact that the software has Peer to Peer elements built in that you cannot avoid using. Basically, if you want to watch the latest episode of Doctor Who, the iPlayer software waits until it has been shown on "real" television and then starts downloading the content for the film not only from the BBC but also from other people who want to watch the same video. Once the video has been downloaded, as long as you don’t switch off your PC or disable the software, YOU will start distributing that content to people who also want to watch the same video but have yet to download it all.

At this point you may well be asking what's wrong with that. Well, there are 2 problems. Firstly it will slow down your internet browsing. Secondly, and of more of a concern, is that it may well cause your ISP to go bankrupt. The problem is all down to bandwidth. It's not cheap. The BBC could have stepped up and said "we will take the financial hit for delivering all this video on demand", removed all the peer to peer elements from their software, and probably discovered an enormous bill at the end of the year for the millions of terabytes of data that they will no doubt have consumed during the day to day running of the service.

What they have done instead is basically pass that fairly expensive buck, without asking them, to your Internet Provider. Nice, huh? So instead of the BBC having to deliver all that content, all those millions and millions of hours of video to all those millions and millions of homes, they are taking all those millions and millions of connected PC's and using their bandwidth instead. For free. The problem is that these costs don’t evaporate into thin air, they simply get passed on to the Internet Providers, who are rightly annoyed at the situation they are being put into.

I'll leave aside the fact that the BBC don’t really explain this in great detail when you download their software. Oh, and if you have broadband connection with monthly download limits, strap in for some hefty bills; leave your PC on with iPlayer installed and you will see what I mean.

So the question is, where will this end? Well, someone has to pay for the service. At the end of the day it will be the end user that pays for it in same manner, be it through increased TV licensing or increased ISP costs. The problem is that this lovely market equilibrium has not yet been reached. The iPlayer has a fairly low installed user base at present, but that will change. Until we get to that equilibrium, expect to see the following things happening:

- ISPs complaining in ever increasing volume about the BBC

- Your internet connection getting slower and less reliable as time goes on

- Your ISP going bust

- Your ISP being bought out

- EVERYONE who has to pay for going over their capped monthly broadband limits getting VERY upset

Personally, I am of the opinion that the BBC should carry the costs of the distribution. It's their content, and it's their service. Offloading the majority of the cost to a private company without their acknowledgement is just not cricket. Maybe this discussion will occur in the House of Commons in the next 5 years, but I won't be holding my breath!

Some links for the interested:

• Upset ISPs
• Upset Consumers
• More Upset Consumers
• What all the fuss is about