Unfortunately, Raygun didn’t turn out to be very useful. We share this feedback with them
Here are the stats at the top of our performance section:
Given how drastically the median and the average differ, it stands to reason that there are some pretty strong outliers. I wish I could know more about those outliers. I wish I could see the distribution of page load times as a graph, and also be able to filter that graph by adding/removing countries, ISPs, or even users based on analytics traits:
- How is page load speed affecting new/returning customers?
- How is page load speed affecting our biggest customers?
- How is page load speed affecting our biggest market (In our case, US/Australia/Singapore customers)
- Is page load speed a function of country? distance?
The answers to these questions can give us insights into whether the problem is on our end, or our customers’ ends. Maybe it’s a page issue, or maybe we just need to spin up an Asian server because it’s a region issue, or maybe it’s an issue with our customer that is out of our control. It’s unfortunate that we can’t see a list of all countries, ranked by load-time:
We get almost no orders from these countries. Aside from the fact that I’m surprised that people from Kazakhstan have such fast internet, it doesn’t help us much. Maybe even a graph over time would be useful, to see if certain countries are improving or not
Here is are the details for our biggest market, the US:
I really wish I could see a distribution of all page-load times, and a distribution of per-user average page-load times.
Is any slowness just because of a few users, or is it endemic to all users in the US?
Is it limited to a specific region or regions?
Which pages are affecting these users the most?
Almost none of the information on this page is actually helpful to me in identifying and correcting speed-related problems.
Thanks for raising these issues. Regarding the documentation, these can certainly be clarified a bit which I will ensure is done for you. For your specific questions:
- Those metrics are human-readable calculated versions of the Browser PerformanceTiming data we receive for every request (https://developer.mozilla.org/en-US/docs/Web/API/PerformanceTiming). Latency is defined as connectEnd - connectStart, e.g the time it took for the transport layer to open it connection (and depends on network quality and can show degradation at the lower layers). Server is the time it took for the request to be computed on the server minus all in-flight network time.
- In general Pulse surfaces data from the Browser Timing API. We currently cannot surface websockets to any acceptable detail as this data is not provided by the browser vendors. If they do surface it we will capture and pass it through. Futhermore and for the other supported assets, long running events will not disrupt latency/server as the former only concerns network conditions and the latter concerns calculated server time for a particular request.
- Render time concerns what the browser records as the pageLoad event time, e.g DOMContentLoaded (when visual components are displayed to the user and the page’s state is ‘complete’ and ready for interaction. If a webpage contains scripts that have an SPA app lifecycle, this use case is not particularly well understood by the browser timing API. In this instance we offer virtual pages to track page load events, and are planning to support custom timing data and custom events in future to support the SPA use case.
- Children will only include child assets that load prior and around DOMComplete, so long-running Ajax calls won’t be considered for the Children metric. These will however be captured and surfaced on the waterfall and in other components e.g Most Requested XHRs.
Thanks for your feedback on the users and sessions components, that is useful. If you wanted to add a feature request for the additional filtering/sorting at https://raygun.com/thinktank/forum/4 that will help as we use that to prioritise upcoming work.
Specificly for your use cases mentioned at the bottom, we are aware of these pain points and are conducting research into increasing the flexibility of Pulse and adding more data and tools that can surface that data and answer those questions intuitively.