Mining Tweet data


#1

I am a geography student. I am interested in mining tweets for geographic data in support of my thesis on the new Geography. I know maps are being developed by some developers. I would like to be able to develop maps myself. How do I do that? What is the process?
Thanks in advance.


#2

The first step is to narrow down the subject - “mining tweets for geographic data” is a pretty broad definition. In general, though, you simply collect tweets with one of the three APIs - Search, REST or Streaming - and extract the geographic data from them. You can then plot the data on a map or do some more complex analysis. It’s pretty straightforward to extract geotagging in most scripting languages. I do it in Perl but Ruby, Python, PHP and JavaScript all have facilities for handling the data that comes back from Twitter in JSON format.

I’ve collected some geographic data using the Streaming “sample” endpoint. Somewhere between 1 and 2 percent of all tweets have geographic data attached. It varies by time of day - when the USA is actively tweeting there are more geotagged tweets than when we’re asleep. :wink: In any event, you’ll only get a sample - not all Twitter users have enabled geotagging, and not all geotagged tweets are tagged down to the POI level.


#3

I realize I was asking a very broad question…I do very much appreciate your patience and your explanation. I am in the planning phase of my thesis. What I notice is that most of the tweet mappers are creating macro maps. My plan is to create more discrete ones (as I think you have surmised). I shall get my ducks in a row and reply back with more specific requests.Again, thanks. Scout


#4

The “locations” method on the “filter” endpoint on the streaming API is probably what you want then.

https://dev.twitter.com/docs/streaming-api/methods

This will let you capture all the tweets coming from inside some geocode-defined bounding boxes. If there are lots of tweets coming from inside your bounding boxes you might receive “limit” messages with counts of dropped tweets, but I’m guessing that won’t happen given the current small fraction of tweets that are geotagged.


#5

Would you please somebody help me , how i can connect OAuth in RStudio to import data from twitter . I already wrote these commends , but i got folloeing error l:
""“Error: unexpected ‘/’ in " requestURL=https:/”

                     		accessURL=htts://api.twitter.com/oauth/access_token,""""""

I wrote these commends : I don’t know what should to do??

library(RCurl)
library(twitteR)
library(ROAuth)

requestURL <- "https://api.twitter.com/oauth/request_token"
accessURL = "http://api.twitter.com/oauth/access_token"
authURL = "http://api.twitter.com/oauth/authorize"
consumerKey = "XXXXXX"
consumerSecret = “XXXX”

Cred <- OAuthFactory$new(consumerKey=XXXXX,
consumerSecret=XXXXXXXX,
requestURL=https://api.twitter.com/oauth/request_token,
accessURL=htts://api.twitter.com/oauth/access_token,
authURL=http://api.twitter.com/oauth/authorize)
credentials$handshake()


#6

well thats easy to say 'cause i can’t get the geo coordinates from twitter api on python, i use twitter.py api http://code.google.com/p/python-twitter/

but if you said its so simple, can you guive me an example to get coordinates from my mentioners please?


#7

Hey there OP,

I know its been a while since you asked this question, but we actually created a site that will allow you to mine twitter for keywords, hashtags, and users. www.twittermining.com

-TwM


#8

Member of BJP ,Followed by India’s next PM Sh. Narendra Modi,MD Visionary Credit & Canvas Infratech,Proud Hindu,Tweets Personal,RTS not Endorsements


#9

Supporter of Narendra Modi (next PM of India). Hate CONgress & Pseudo-Sickularists. Mission 272+. CONgress Mukt Bharat. NaMo Namaha. Jai Hind. Vande Mataram.


#10

new PM give hope to country "Acche din aane wale hai now people are feeling good day is around the corner


#11

i am new to python. please provide me the script to download the data from twitter . Thanks in advance


#13