Service degradation over the past weeks

The last week hasn’t been great in terms of availability of our data feeds. We’ve had numerous outages that are outside our control:

  • On 3rd June, we had an outage of all our data feeds from 0320 to around 0333. This was followed by a period of high latency (the delay between a message being generated by a train describer and it being received by our systems – sometimes known as lag) between 0446 and 0512, then from 0518 to 0524. From 0631 to around 0735, data was being received by our systems between 15 seconds a 2 minutes late.
  • On 4th June, we had an outage of all our data feeds from 2342 (on 3rd June) to around 0912. This resulted in our live track diagrams freezing for the duration of the outage, and for some hours afterwards, maps showing out-of-date data and the location screens not reporting any train movements.
  • On 5th June, we had an outage of all our data feeds from 0941 to 1103 – again resulting in our maps showing out-of-date data for some time after the feeds returned, due to the nature of the feeds.
  • On 6th June, we had a partial outage on our train movement data feed from 1740 until around 2115 on 7th June. Our monitoring systems should have picked up this issue but didn’t – we are investigating why. On the same day, from 1747 to around 2130, we had an outage of all our our data feeds – both feeds returning to normal afterwards, but again resulting in our site showing incomplete and out-of-date data.
  • On 7th June, we had an outage of all our data feeds from 0846 to 0851, although the maps will have updated quickly afterwards due to the volume of trains running in the end of the morning peak period
  • On 8th June, we had an outage of our train describer feed between 1155 and 1201, from 1215 to 1311, from 1314 to 1328, from 1343 to 1350, from 1358 to 1407, from 1415 to 1430, and from 1438 to 1440. During this time, the train movement feed was operating normally, but this issue caused our maps to freeze repeatedly and intermittently for nearly two and a half hours.
  • On 9th June, we had an outage of all our feeds between 1320 and 1350, a partial outage between 1428 and 1440, then a full outage from 1440 to 1453, followed by a further outage between 1601 and 1620.

Quite rightly, many of you have voiced your concern over our inability to provide you with a useful service over the past week. We have no control over the Open Data feeds, which are available to anyone at https://datafeeds.networkrail.co.uk/, and we are not happy at the huge number of outages and the combined total downtime over the past week. These issues have been happening for months, if not years, with seemingly no resolution in sight.

We are facing a difficult decision: do we continue to use the public feeds to operate the OpenTrainTimes public website, or do we move to more reliable feeds which aren’t available to, or suitable for, many other developers and websites to use – since they’re not quick to set up, and not free to use when you consider the additional hardware and technical work required. The answer to this isn’t simple, and we continue to hope that the Open Data feeds will be fixed so that everyone can benefit from them.

3 thoughts on “Service degradation over the past weeks

  1. What are Network Rail’s obligations to provide the feeds? Would it help if end-users of sites like yours complained to them about outages, or would that be counterproductive?

  2. Good morning Peter. I’ve just joined Network Rail as Group CIO and I’m sorry that you have had this experience and continue to do so. There has been work done, and more work ongoing to improve the situation, and we will continue to improve. More than that, I would like to engage in a collaborative working arrangement with yourself and other partners in the rail ecosystem who rely on our data, so we can jointly share our experiences and work together to address the challenges we face. Please do get in touch with me at your convenience, thank you.
    Aidan Hancock

  3. Please continue to use the free feeds
    We have to accept that as we’re not paying for it the site sometimes has issues. I’d rather accept this than have to pay for something. You also can’t guarantee that if you do use other feeds that they will be better.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.