Web Projects

I recently participated in an online discussion with someone working on a web application. I do that a lot. It’s a testament to the fact I need to get out more. But anyway, part of the conversation went roughly as follows:

Me: “Will it scale?”

Web Developer: “Oh yes, it’s based in the cloud. We can add nodes really easily.”

Me: “That’s not really answering what I asked.”

Web Developer: “We can handle millions of users no problem.”

Me: “Can you? Really?”

It turned out that he couldn’t. The application in question built a network of links between users with similar interests. He had considered the problem of what happens when 100,000 people want to access the application at once and solved it by rolling out a solution on a cloud provider’s network. But that was only solving half the problem, and it was the easy half at that. When his application actually had 100,000 test users it kept falling over because creating the networks took far too long. It was too complicated.

An error that many web developers are make when it comes to cloud technology is mistaking horizontal scalability for vertical scalability.

Horizontal scalability is the number of things that your application can do at once. It’s parallelism. If you need to serve pages to thousands of people at once then having a horizontally scalable architecture is great. Fortunately, these days horizontal scalability is trivially easy to accomplish. It’s one thing cloud computing solves. If you need to have hundreds of servers all running little scripts, brilliant, you’re sorted.

Vertical scaling is far more complicated though. It’s, in essence, the problem of having a huge amount of data that a single script needs to access to do it’s job. To scale vertically the script itself needs to be clever; it won’t be able to handle more data simply by chucking more hardware at it. This is where cloud computing begins to fall down. By putting his solution in the cloud, my developer friend lost all the advantages that enterprise databases such Oracle give you. Most cloud providers afford you a fast, simple key-value store and leave you to code the clever stuff yourself. If you’re dealing with massive datasets that isn’t going to work unless you’re a rather talented coder. There’s some progress being made by services such as Amazon’s SimpleDB that can index your data automatically, but it’s a long way behind even a database like MySQL.

So what’s the solution? Fortunately it’s a very easy problem to solve. Don’t rush out to the cloud thinking it’s the be all and end all of scalability problem solving. What your code is doing is as important as the amount of hardware running it. Some applications are well suited to running ‘out there’, but some aren’t. Recognise which your application is and code it accordingly.

If your next mobile phone has a whopping 64GB of storage space, a high resolution touch screen, and all the usual things like GPS, 3G, Wifi, more apps than you can shake a particularly large stick at, and so on you’d probably be over the moon. Who wouldn’t want such a device? What if it was 25cm tall, 19cm wide and weighed nearly 3/4 of a kilogram? That’s not so good.

There are going to be lots and lots of devices along those lines viewing our websites soon though. What I’m describing is Apple’s new iPad, and the specifications for the raft of “mobile tablet” computers that will follow are likely to feature very similar details.

What this means for a developer is that life is going to be even more complicated than it is now. Already tasked with producing HTML, CSS, and JS code that looks snazzy and awesome in Firefox 3, Internet Explorer 8, and Opera 10, and degrades to something less snazzy and awesome in Firefox 2, Internet Explorer 7, and Opera 9, plus works properly in Safari and Chrome, and if you’ve paid attention to my previous post at least, is also tailored to mobile phone browsers, a web designer/developer now has to target a new device that’s part way between phone and laptop.

Thankfully the iPad version of Safari is a relatively well known target platform as it conforms to the same spec as the iPhone version, with a couple of very minor differences.

First of all, the User Agent string is different. This is obvious really, and if it weren’t it’d make life pretty much impossible. The new string is;

Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B334b Safari/531.21.10

The ‘iPad’ string makes detection trivial. For example, if you’re using JQuery (as you should be);

jQuery(document).ready(function($){
var deviceAgent = navigator.userAgent.toLowerCase();
var agentID = deviceAgent.match(/iPad/);
if (agentID) {
//iPad specific code
}
});

Similarly it’s as easy on the server site. A PHP example using the browsercap.ini functionality (you’ll need an up-to-date browsercap.ini file from http://browsers.garykeith.com/downloads.asp);

if(ini_get("browscap")) { $browser = get_browser(null, true); print_r($browser); }

The $browser array should then tell you that you’re sending content to an iPad that you can then tailor accordingly.

That’s how to tell you’re sending a page to an iPad, but that doesn’t tell you the capabilities of the device. Obviously the first issue is that Flash content won’t be displayed. At the very least you need to use JavaScript to degrade Flash player objects to an HTML equivalent. HTML5′s <video> and <audio> tags will work, otherwise dropping down to a static image is preferred.

The next issue is with CSS fixed position elements. Ordinarily, on a desktop browser, a post ion: fixed element stays wherever you’ve told it to stay. That’s very useful for static navigation, ‘feedback’ side tabs, etc. Unfortunately though, on a mobile device that allows the user to zoom in on things, the browser doesn’t know whether something that’s been defined to be displayed “20px from the left” should be there only at the top level of zoom, all the time, or something in between. The solution to this is through the use of viewports. On the iPhone and iPad the it’s the viewport rather than the window that’s zoomed it. Consequently, if you need your element to remain fixed in position you need to attach it to a position in the window rather than the viewport.

The final common issue is with Javascript events. onMouseOver, onMouseOut and the CSS pseudo class :hover will rarely be seen by a user on a touchscreen device. Instead you should replace those actions with touchstart and touchend.

There are a few more things to consider, but they’re strictly edge-case scenarios that won’t affect most websites. If you want the full lowdown you should take a look at Apple’s iPad development Tech Note: http://developer.apple.com/safari/library/technotes/tn2010/tn2262.html

The past few days of Twitter activity with people complaining about the effectiveness of their MP, with so many failing to turn up to an important vote, and many more not putting in an appearance at the preceding debate, coupled with the apparent lack of visibility to people writing letters and emails to them has reminded me of a project I started a couple of years ago that was designed to track the work that MPs do in a sort of league table.

GovWatchUK (working title) was to be a crowdsourced opinion engine that took a number of statistics from open data sources such as MySociety, DirectGov, Twitter, RSS feeds and Google News, and blended them together into a set of effectiveness metrics – rather like KPIs for MPs. The social element was to take news items from an aggregator and let people rate them on a scale of positiveness from -5 to +5. If an MP was getting favourable news and they were working hard in government then they’d move up the leaderboard, if the general opinion was negative they’d move down.

The project got as far as a front end design, database, and hooks into MySociety, but then other projects took over and it fell by the wayside. I’ve gone back to it a couple of times since but never completed it.

With the looming general election I wonder if it’s not time to resurrect the project to see if I can track the rise and fall of MPs over the parliamentary term.

Here’s a scary admission, one that I hope won’t make you turn and run: I used to sell double glazing for a living. It was a long time ago, I was young and naive and I didn’t know any better, I was barely out of school. What did I know?

While I was working as a salesperson, tramping around damp Bournemouth housing estates knocking on doors and being turned away more often than I’d have liked, I learnt a few things about selling. The biggest lesson was that I definitely didn’t want to do it for a living, but besides that I learnt that people are actually incredibly receptive to sales messages so long as you pitch them properly. If you respect a person’s point of view, if you’re polite and courteous, and if you’re not aggressive, then most people will at least listen to what you have to say before closing the door and complaining about people knocking on the door at all hours.

How does this apply to the web?

One thing that has always struck me as a little odd is the distinct separation between website technical people, the developers and designers who make things happen, and the sales people in charge of marketing products to the customer. Techies, on the whole, tend to shy away from any sort of overt marketing on their websites, preferring to do things that they believe won’t annoy or irritate the customer.

The problem that I’m leading to here is simple – things that we think would annoy the customer very often don’t. People aren’t all turned off by the same things. And so long as your message is delivered in a courteous and polite way people will usually sit and read it before leaving. If you’re a technical person, and you don’t have any sales experience, you probably shouldn’t be the one deciding on the marketing messages on a website.

Case Study: Ling’s Cars

Ling’s website, on first viewing, looks a lot like someone vomited it on to your browser. It’s harsh, brash and garish, it flashes and it scrolls. If you talk to a techie about it you’ll hear talk of how it looks old, out-of-date and awful.

Despite this apparent design disaster the website works. Ling makes a living from her business, and employs 5 staff. It might seem counter-intuitive from the perspective of a developer, but that’s because, and this might be a harsh reality, developers don’t always know how to promote a business. Ling’s publicity message is that of “cheap!”. Her leasing prices reflect that. A crisp, beautiful website would give the user the wrong impression.

To draw an analogy to the high street, Ling is the equivalent of Primark. If a typical Primark customer walked in to find beautiful, tidy shelves filled with neatly folded clothing, lit with soft white pearl light and Mozart playing in the background as you might find in a high-end boutique, they’d turn around and leave. Their first impression wouldn’t be “isn’t this lovely”, it’d be “I can’t afford this”. Your website needs to reflect the needs and wants of the customer. If the customer wants cheap then it’s got to look cheap.

So how do we fix this problem?

Ideally all web developers should have some sales experience but that’s not a practical solution. Instead I suggest that if you’re a web developer you should try reading some books about selling, about marketing, and about how customers think. Customers are a strange breed who don’t appear to think like rational human beings some of the time. If you’re a client commissioning a website you need to make sure that your web development company understands the way you do business – do not assume that they will understand who you sell to and how your customers like to be addressed. You’re the expert in the way that you do business. Communicate that expertise to the people working for you.

What this doesn’t mean is that web developers and designers should ignore their experience and just do whatever the client asks. There are things that don’t work online that do work offline. The development of a new website needs to be a collaboration between the client and the developer.