Pairwise Correlations in HTTP Archive: Pearson or Spearman ? Part-1

In our group,  we have been interested in the question of how the structure of a webpage influences its performance on the web. It is (in my opinion) one of the key questions at the heart of distributed web application delivery. Thanks to amazing resources like HTTP Archive and BigQueri.es, it is pretty straight forward to access and play with large scale web performance data (measured twice a month across 400,000+ websites and made available for free!). 

Probably the simplest (if not the best) model one could consider in studying structure-performance relationship of webpages is to look at the pairwise correlations between various measures of webpage structure (like number of images, amount of image or JavaScript bytes etc.) to various page-level performance metrics (like startRender, PageLoadTime / onLoad, Speed Index etc.). HTTP Archive has a nice set of visualizations on the Interesting Stats page that provide this information for a few chosen metrics. While going through these plots, Clark and I started to wonder what correlation measure HTTP Archive actually uses and whether it is an appropriate choice. I wrote to Steve Souders (the creator of HTTP Archive) and he responded very quickly with links to the code and some associated information. The code, and our own calculations  (using public HTTP Archive datasets) suggest that Pearson correlation is used in HTTP Archive for measuring pairwise structure-performance relationships. See the plots from Clark below, so that you can see for yourself.

While Pearson correlation is quite handy when its assumptions are satisfied, it is not uncommon for real-world datasets to break these assumptions. It will be worth checking if Pearson correlation is a suitable measure for HTTP Archive data.