I am a master student and I have been approved in application for an academic access, but I can not have access to historical data since 2015 in a large quantity. I tried several codes and methods but it sounds like that my access is not an academic one. In my developer account above the project elevated access is written, but still I could not gain the required data. Why that happened and what can I do, is it probable that there is some problem with twitter API? can anyone provide the link for all the steps required after approval for extracting historical data in a large quantity?
What exact approach did you try?
In my developer account above the project elevated access is written
This means you do not have Academic access, unless you have another project, where it says For non-commercial use only [ACADEMIC] beside it.
With Elevated Access you can use v1.1 APIs. However, this may not be useful to you for large quantities of data - even with the v1.1 Premium API it’s hard to get a lot of data, but it’s possible.
I would recommend grabbing an existing data set,
and hydrating it with twarc for example: twarc2 (en) - twarc
See here on getting started:
Thanks for your response. Would you please help me understand how I can get academic access? As I told you before, I made a request for academic access and twitter sent an approval email after about two weeks but still I do not have access to it. I do need an academic one and I do not know how I can get it with certainty.
If you’ve applied and got an approval, you should have had a new project created for you on https://developer.twitter.com/en/portal/projects-and-apps with “Academic Research”
However, if you applied to be a Developer as an Academic, this does not automatically grant you academic access - it’s a separate application: Twitter Developer Access - twarc
It is surprising that I have not got the academic access due to the fact that I have gone through the entire process. I will try it for for another time. Thanks for the helpful links.
My main challenge is that I need to know the users’ country beside the text of tweet. The only way to have access to the country of users is through academic API. Am I right? I mean the country the users yielded their information with when they signed up.
Not all tweets have geo location information, so it’s not always reliable. The Location field in the user object is something each user can enter anything into, so it’s not a reliable location either.
Elevated Access should be enough to use the Geo operators on the filtered stream: More operators supported with Twitter API v2 filtered stream | Twitter Developer Platform but NOT on Search. So you can get new tweets as they’re created, but not search for old ones if you only have Essential Access.
This exact thing is not available anywhere - there is no way to find this out, the only location information available is the Place objects in the tweet metadata. The metadata is available in any access level, you don’t need academic access at all to request it in tweets Place objects | Docs | Twitter Developer Platform - but you just can’t query for it without academic access.
Hope that helps!
I greatly appreciate your help.
Since I need to extract approximately 1 million tweets since 2015 until 2021 with place object; what is the best approach?
I also need to know the numbers of retweet per each tweet prefereably.
Is there any feasible approach other than academic access?
Sure - 2015 to 2021 with geo information is a very broad set but I think without academic access, your best bet is to pick an existing dataset from https://catalog.docnow.io/ and hydrate that, the data should contain the retweet counts in there - but geo information depends on the dataset.
but the query I am looking for is not available on the site you mentioned. what are the other approaches?
Oh, i just remembered - the next best thing may be https://tweetsets.library.gwu.edu/ that will potentially let you create a dataset from existing datasets that match your query - but this is limited to the data that they already have.
Otherwise, using the v1.1 Premium API, either the very limited free sandbox calls or Paid premium is the only other way to get old data. Search API | Twitter API | Docs | Twitter Developer Platform
So, I should pay for that!
I appreciate your guidance.
