I am currently using the COVID-19 streaming endpoint and just discovered that due to a failure in an email error notification, I’ve lost a 2 month worth of data. Is it possible to still get this lost data somehow?
Possibly - i’ve also had a few outages in the stream that i would like to fill in.
I have not implemented anything yet but i suspect it should be possible to do using the v2 academic Access endpoint to retrieve the tweets using context annotations GET /2/tweets/search/all | Docs | Twitter Developer Platform and then hydrating the tweet ids with v1.1 status lookup endpoint to recover the same tweet format and keep it consistent.
What exact time range did you miss? I may be able to extract the IDs for you - and you can use something like twarc to hydrate the IDs and get the data that way. (But like i said, i also have outages and missed things due to various reasons)
Thank you for your reply! The time range is April 17, 15:50 UTC to June 16, 12:25 UTC. I’d really appreciate if you could extract the IDs, I can hydrate no problem.
I haven’t tried the v2 academic endpoint yet, but maybe a good time to look around.
1 Like
Unfortunately it seems i’m also missing a large chunk of data between those ranges!
I could get
2021-05-31 11:00 UTC
to
2021-06-16 23:00 UTC
It’s still processing but if you DM me an email i can send you the link to download the IDs.
Maybe someone else on https://www.docnow.io/ slack can help fill the gaps.
1 Like
