Six months ago, our criteria for developing apps using Client Side MVC(Single Page Apps or SPA) or Server Side MVC was either: Client side MVC if SEO (search engine optimization) was of no concern or Server side MVC if it was…
This strategy however has proven less applicable as SPA’s popularity grows and server side MVC continues its decline. We often run into projects that need both client side MVC and SEO. Since most search engines do not execute JavaScripts on the html pages and content on SPA pages are mostly generated by JavaScript, indexing Single Page App pages yields bad SEO results.
The SEO Solution
Using (our favorite) AngularJS, here is one approach to solving the SEO problem for Single Page Apps. Note: The same approach can be used for other frameworks (e.g. Backbone and Ember) as well. See links under Resources for more details.
Most search engines support the hashbang URL convention – when they see #! in a url, they will substitute it with ?_escaped_fragment_ instead.
For example:
http://example.com/#!/products will be indexed at the expanded url:
http://example.com/?_escaped_fragment_=/products So the idea is to store and serve the pre-rendered pages (snapshots) at the expanded urls. The snapshot of a page contains the DOM content already rendered out by JavaScript.
It takes several steps to make this work:
1. Enable hashbang on the client side
Since by default Angular routes (urls) uses the /#/ prefix, we need to use the following to make routes to use the /#!/ hashbang format.
$locationProvider.hashPrefix('!'); For apps that use the HTML5 pushState, we will need to add
<meta name="fragment" content="!"> to the <head> section of the to-be-indexed pages. In Angular apps, push state is enabled with:
$locationProvider.html5Mode(true); 2. Route urls with escaped_fragment to cached pages on the server side
Rails/Sinatra - use Rack middleware to redirect to pre-rendered urls Node/Express - use middleware to redirect to pre-rendered urls Apache - use mod_rewrite to rewrite to the pre-rendered urls Nginx - use proxy to the pre-rendered urls For Rails apps, you can determine in the Rack middleware if a request is coming from the crawler by checking if ‘_escaped_fragment’ is present on the request and redirect to the pre-rendered urls:
query_hash = Rack::Utils.parse_query(request.query_string) if query_hash.has_key?('_escaped_fragment_') # redirect to the pre-rendered urls ... end 3. Capture, store, and cache the pre-rendered snapshots
Do this on a regular basis, preferably as soon as content changes are made to the pages that need to be indexed. Note: Only capture SEO worthy pages.
There are a number of tools that can be used to take snapshots of web pages. PhantomJS, CasperJS, and Zombie are great options.
Here’s an example of JavaScript that uses CasperJS (which uses PhantomJS and provides some nice higher level browser functions) to capture the page:
var casper = require('casper').create(); var url = 'http://localhost:3000/#!/'; casper.start(url, function() { var js = this.evaluate(function() { return document; }); this.echo(js.all[0].outerHTML); }); casper.run(); You can parameterize something like this, and run it against all the pages that need to be indexed by the search engines.
Here are the screen shots of the Angular TodoMVC app (with Rails backend):
Screen Shot #1 – Original AngularJS rendered content 
Screen Shot #2 – As crawler sees it, no content was indexed (simulated with JavaScript turned off in the browser) 
Screen Shot #3 – Snapshot taken with CasperJS, served at the expanded url with all the content 
As you can see, the snapshot (Screen Shot #3 above) has all the content from the original page (Screen Shot #1) sans styling, ready to be indexed by the crawlers.
SPA SEO Services
Not interested in the DIY method? Here are a few services that provide SEO for Single Page Apps:
- Prerender.io
- Brombone
- SEO.js
If you use Divshot (you should take a look if not), SPA SEO support is built in, they partnered with Prerender.io to bring you this nice feature. See the details here:
- Divshot SEO for Single Page Apps
