Collect tweets for specified time using twitter4j

[Dhuberry]

Hi,
I am currently, working on Twitter sentiment analysis project. I collected tweets using streaming API based on topic. But, I want to run my streaming API for 1 hour to collect tweets. Is it possible to collect tweets on preferred time.
For example, topic is “beer” and for 1 hour to collect. Like cronjob.

[andypiper]

You could do that using something like cron4j. There’s nothing specific to Twitter here, unless I’m mistaken?

[Dhuberry]

Thank you so much. I have to collect tweets over the weekend (say from Friday night to Monday morning - nearly 48 hours), so its possible to collect live tweets(streaming API) without rate limit rite? I red the documentation, it says public streams has no no rate limit. We do have limits for REST API.

-Prabha.

[andypiper]

I red the documentation, it says public streams has no no rate limit.

Can you please show me specifically where that is stated?

The Streaming API is limited to 1% of the total volume of all Tweets at a point in time, and you may only have a couple of connections, and must not reconnect too quickly. Additionally, you are limited to the number of filter terms that you can use.

[Dhuberry]

Hi,
My apologize, I heard it from my community people. Later I red the document and got confused with the limits. Now I am quite clear with rate limits for streaming API. I am trying to collect UK election poll tweets tomorrow, so is that possible to run the streaming API from morning 7 to night 10 continuously? If so then I can set cron4j for this job.

[andypiper]

Assuming your code doesn’t disconnect, and you have an appropriate set of filter terms in mind, then that should work just fine.

[Dhuberry]

Thank you so much!!!

[Dhuberry]

Hi,
While working on streaming API, most of tweet’s geo location were null. If it was null, how do we get location. Also, I tried with multiple filter query option ( location and keywords) but I got either the tweets based on location or based on the keywords. But I want to get tweets based on the location and those tweets should have the keywords.

String keywords[] = {"health", "politics"};
double[][] loc={{-122.75,36.8},{-121.75,37.8}};
fq.locations(loc);
fq.track(keywords); 

Help please!!!

[andypiper]

Only a very small percentage (~1-2%) of Tweets have geo data attached, so this is not completely unexpected.

[Dhuberry]

Is it possible to collect tweets in a time interval. Something like this,
String modifiedDate= new SimpleDateFormat("2016-05-09 08-00-00").format(date); String modifiedDate2= new SimpleDateFormat("2016-05-09 12-00-00" ).format(date); query.setSince(modifiedDate); query.setUntil(modifiedDate2);
I tried but no luck.

[andypiper]

It is possible to use date queries, but not time queries. You’d need to filter the time range yourself out of the main results. Full search API information is here.

[Dhuberry]

Thank you so much!!!

[Dhuberry]

In streaming API, lots of duplicate tweets were linked like multiple copies of same tweet. I cant overcome from this issue.
Also is it possible to collect only original posts using streaming API?

[andypiper]

You’d have to filter that for yourself, there’s no parameter that will take them out for you.

[SocialOpinions]

Hi,

I’m building a tool called Social Opinion that does exactly what you need

You can login with your existing Twitter handle and create a “Campaign”, supply some keywords and the tool will collect tweets 24x7 in the background for you and plot the sentiment or each tweet.

It also has rich charting features such as a real-time line graph and bar/pie charts. It’s in beta and I hope it can help you!

Meant to say, it also can send you email alerts depending on certain thresholds such as X number of tweets have been found or when the sentiment has spiked/dropped.

You can find it here: www.socialopinion.co.uk
@socialopinions

Regards,
Jamie.

[andypiper]