Search API & the new complete tweet index announcement?

[karim_ouda]

Regarding this announcement

Building a complete Tweet index

When we will be able to access the new historical index using the REST API ?

[riverfold]

I’d love to hear an official statement from Twitter on this as well. Otherwise we’ll have to assume that restricting the new search to only the official Twitter apps is just another way to artificially limit the usefulness of third-party apps.

We’ve wanted this feature for years (and I’ve build a couple apps specifically focused on working around the limited search index). It’d be great to have it available to all apps.

[edsu]

I would also love to hear if the search REST API call could support accessing historical data. I think it would be a huge win for social media research to have this functionality available. The search/tweets API call currently supports since and until parameters, but you don’t seem to be able to use them to search earlier than the last week.

As more and more of our culture and politics are reflected in and by Twitter it becomes more and more important to be able to access this historical information in a programmatic way. I doubt many citizen scholars can afford a Gnip account, at least in its current configuration. Perhaps not many traditional researchers can either. It seems a rate limited way of getting at this historical data is in the interests of all parties.

[karim_ouda]

I can’t agree more on the effect on “social media research”

[isaach]

We don’t have any plans to make the full historical index available for free via the public REST API. The focus of search/tweets (and the streaming statuses/filter endpoint) will continue to be real-time search.

The commercial REST API from Gnip currently offers a full-fidelity 30-day index, and it’s possible that we’ll expand that over time. Gnip also offers the asynchronous Historical PowerTrack commercial product, which supports sophisticated multi-predicate search over the entire historical corpus.

For scholars we have a separate program, including the Twitter Data Grants initiative, to make data available to researchers.

[IgorBrigadir]

Thanks!
I guess it makes sense, otherwise Gnip et al would be put out of business… Any idea when the next run of Data Grants will be?

[edsu]

@isaach Thanks for the reply, it’s good to know that I shouldn’t be waiting for the Twitter API to reflect the new index.

I’m a bit bummed, considering Twitter could choose to distinguish Gnip’s API based on higher rate limits, and PowerTrack which people would continue to pay for bulk access, and improved search.

[isaach]

I can certainly understand your disappointment.

[andypiper]