No Tweets in Twitter4J Streaming API example

[andypiper]

Continuing the discussion from Now Resolving: No tweets through twitter streaming api:

Now Resolving: No tweets through twitter streaming api

I’m new in this issue when i run codes which i have copied from web, it takes 20-30 min to get a tweet from stream api. I just downloaded some codes to run as a starter but it is too slow and sometimes gives me error. Do I have something wrong in the following code ?

package twitterstreamexample;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.StatusLine;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
import org.json.simple.JSONValue;
import twitter4j.*;
import twitter4j.conf.ConfigurationBuilder;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.HashSet;



public class TwitterStreamExample {
 
  public static String readUserLocationFeed(String address) throws IOException {
    StringBuilder builder = new StringBuilder();
    HttpClient client = new DefaultHttpClient();
    HttpGet httpGet = new HttpGet(
        "http://maps.google.com/maps/api/geocode/json?latlng=" + address
            + "&sensor=false");
    try {
      HttpResponse response = client.execute(httpGet);
      StatusLine statusLine = response.getStatusLine();
      int statusCode = statusLine.getStatusCode();
      if (statusCode == 200) {
        HttpEntity entity = response.getEntity();
        InputStream content = entity.getContent();
        BufferedReader reader = new BufferedReader(new InputStreamReader(
            content));
        String line;
        while ((line = reader.readLine()) != null) {
          builder.append(line);
        }
      } else {
        System.err.println("Failed to download file");
      }
    } catch (ClientProtocolException e) {
      e.printStackTrace();
    } catch (IOException e) {
      e.printStackTrace();
    }
    return builder.toString();
  }

  public static String getUserLocation(String lat, String lon) throws IOException {
    String userLocation = null;
    String readUserFeed = readUserLocationFeed(lat.trim() + "," + lon.trim());
    try {
      JSONObject strJson = (JSONObject) JSONValue.parse(readUserFeed);
      JSONArray jsonArray = (JSONArray) strJson.get("results");
      JSONObject jsonAddressComp = (JSONObject)jsonArray.get(1);
      userLocation = jsonAddressComp.get("formatted_address").toString();

    } catch (Exception e) {
      e.printStackTrace();
    }
    return userLocation;
  }

  public static void main(String[] args) throws InterruptedException {
    //set the OAuth params from 'https://dev.twitter.com/apps/new'
    ConfigurationBuilder cb = new ConfigurationBuilder();
    cb.setDebugEnabled(true);
    cb.setOAuthConsumerKey("");
    cb.setOAuthConsumerSecret("");
    cb.setOAuthAccessToken("");
    cb.setOAuthAccessTokenSecret("");

    TwitterStream twitterStream = new TwitterStreamFactory(cb.build()).getInstance();
    StatusListener listener = new StatusListener(){

      @Override
      public void onStatus(Status status) {
        /**
         * Initializes every time it receives a tweet
         */
        User user = status.getUser();

        // gets Username
        String username = status.getUser().getScreenName();
        // user location
        String profileLocation = user.getLocation();
        // place
        Place place = status.getPlace();
        String placeCountry = place.getCountry();
        String placeStreetAddress = place.getStreetAddress();
        String placeName = place.getName();
        String placeId = place.getId();
        String placeFullName = place.getFullName();
        String placeUrl = place.getURL();
        String placeCountryCode = place.getCountryCode();
        // tweet id
        long tweetId = status.getId();
        // tweet content
        String content = status.getText();
        // get tweet hash tags
        HashtagEntity hashTags[] = status.getHashtagEntities();
        // geo-location
        GeoLocation geoLocation = status.getGeoLocation();
        double latitude = geoLocation.getLatitude();
        double longitude = geoLocation.getLongitude();
        String userLocation = null;
        try {
          userLocation = getUserLocation(String.valueOf(latitude), String.valueOf(longitude));
        } catch (IOException e) {
          e.printStackTrace();
        }

        System.out.println("UserName: " + username);
        System.out.println("UserLocation: " + profileLocation);
        System.out.println("TweetId: " + tweetId);
        System.out.println("Tweet: " + content);
        HashSet<String> hashTagContainer = new HashSet<String>();
        for(HashtagEntity hashTag: hashTags) {
          hashTagContainer.add(hashTag.getText());
        }
        System.out.println("HashTags: " + hashTagContainer);
        System.out.println("Lat: " + latitude);
        System.out.println("Long: " + longitude);
        System.out.println("User Full Location: " + userLocation);
        System.out.println("Place_Country: " + placeCountry);
        System.out.println("Place_Name: " + placeName);
        System.out.println("Place_FullName: " + placeFullName);
        System.out.println("Place_ID: " + placeId);
        System.out.println("Place_StreetAddress: " + placeStreetAddress);
        System.out.println("Place_URL: " + placeUrl);
        System.out.println("Place_CountryId: " + placeCountryCode);
        System.out.println();
        System.out.println();
      }

       @Override
      public void onDeletionNotice(StatusDeletionNotice statusDeletionNotice) {}

      @Override
      public void onTrackLimitationNotice(int i) {}

      @Override
      public void onScrubGeo(long l, long l2) {}

     @Override
      public void onStallWarning(StallWarning stallWarning) {}

     @Override
      public void onException(Exception e) {}
    };

   
    FilterQuery fq = new FilterQuery();
    
    //Keywords to filter the stream on
    String keywords[] = {"FENERBAHÇE"};
    fq.track(keywords);
    
    twitterStream.addListener(listener);
    twitterStream.filter(fq);

   Thread.sleep(50000);
   twitterStream.cleanUp();
   twitterStream.shutdown();
  }
}

[andypiper]

@mustafakya I haven’t run your example myself, but one thing I wonder about is how often your filter string “FENERBAHÇE” is likely to occur in the Twitter stream. Have you tried other filter terms to see whether the code also takes a long time to receive any Tweet data?

Also, you say this sometimes gives you an error - what is the error code and output that you are seeing?

[mustafakya]

Thank you Mr Piper, i solved my problem but now i have another one. I want to make search for old tweets. But Twitter Api gives chance to search for recent tweets or 7-8 days old. How can i use time interval to do that withoung paying? and also how can i use twitterStream and filtered query with since and until

[andypiper]

You can use the search/tweets endpoint to retrieve Tweets up to seven days old. Beyond that, if you need 30 day or even full historic search, there are paid enterprise products from Gnip.

Alternatively, you can listen on the streaming API, but that will be in realtime so there’s no concept of filtering on since and until.

[pavankothagorla]

Hi @andypiper

I am using search API to collect live tweets for my academic research. I am facing this problem from last 2 hours. Can you kindly let me know if the service is broken?

[andypiper]

I am facing this problem from last 2 hours.

which problem?

If you have an issue different to the one in the topic or first post in the topic, please open a new one. We cannot help you if you do not explain in detail what your issue is.