Considering the example mentioned on your GitHub project (https://github.com/twitter/hbc/blob/master/hbc-example/src/main/java/com/twitter/hbc/example/FilterStreamExample.java), lets take a scenario:
I have 3 users: User A, User B and User C
User A: wants all the tweets for "Brazil"
User B: wants all the tweets for "Germany"
User C: wants all the tweets for “San Francisco”
From example on Github
BlockingQueue queue = new LinkedBlockingQueue(10000);
StatusesFilterEndpoint endpoint = new StatusesFilterEndpoint();
// add some track terms
endpoint.trackTerms(Lists.newArrayList(“twitterapi”, “#yolo”));
Authentication auth = new OAuth1(consumerKey, consumerSecret, token, secret);
// Authentication auth = new BasicAuth(username, password);
// Create a new BasicClient. By default gzip is enabled.
Client client = new ClientBuilder()
.hosts(Constants.STREAM_HOST)
.endpoint(endpoint)
.authentication(auth)
.processor(new StringDelimitedProcessor(queue))
.build();
// Establish a connection
client.connect();
Now if I need a different messageQueue for each user, I will have to create 3 client and connect 3 times. When I do that, I see error as
[0m[0m16:04:06,946 INFO [com.twitter.hbc.httpclient.BasicClient] (default task-6) New connection executed: tweetStream-client, endpoint: /1.1/statuses/filter.json?delimited=length&stall_warnings=true
[0m[0m16:04:06,947 INFO [com.twitter.hbc.httpclient.ClientBase] (hosebird-client-io-thread-0) tweetStream-client Establishing a connection
[0m[33m16:04:07,660 WARN [com.twitter.hbc.httpclient.ClientBase] (hosebird-client-io-thread-0) tweetStream-client Error connecting w/ status code - 420, reason - Enhance Your Calm
[0m[0m16:04:12,662 INFO [com.twitter.hbc.httpclient.ClientBase] (hosebird-client-io-thread-0) tweetStream-client Done processing, preparing to close connection
[0m[0m16:04:12,665 INFO [com.twitter.hbc.httpclient.ClientBase] (hosebird-client-io-thread-0) tweetStream-client Establishing a connection
[0m[31m16:04:12,665 ERROR [com.twitter.hbc.httpclient.ClientBase] (hosebird-client-io-thread-0) tweetStream-client Unknown exception while establishing connection to https://stream.twitter.com/1.1/statuses/filter.json?delimited=length&stall_warnings=true: java.lang.IllegalStateException: Connection pool shut down
at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:167) [httpcore-4.2.4.jar:4.2.4]
at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:199) [httpcore-4.2.4.jar:4.2.4]
at org.apache.http.impl.conn.PoolingClientConnectionManager.requestConnection(PoolingClientConnectionManager.java:188) [httpclient-4.2.5.jar:4.2.5]
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:448) [httpclient-4.2.5.jar:4.2.5]
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) [httpclient-4.2.5.jar:4.2.5]
at org.apache.http.impl.client.DecompressingHttpClient.execute(DecompressingHttpClient.java:137) [httpclient-4.2.5.jar:4.2.5]
at org.apache.http.impl.client.DecompressingHttpClient.execute(DecompressingHttpClient.java:108) [httpclient-4.2.5.jar:4.2.5]
at com.twitter.hbc.httpclient.RestartableHttpClient.execute(RestartableHttpClient.java:86) [hbc-core-2.0.2.jar:]
at com.twitter.hbc.httpclient.Connection.connect(Connection.java:44) [hbc-core-2.0.2.jar:]
at com.twitter.hbc.httpclient.ClientBase.establishConnection(ClientBase.java:179) [hbc-core-2.0.2.jar:]
at com.twitter.hbc.httpclient.ClientBase.run(ClientBase.java:142) [hbc-core-2.0.2.jar:]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_05]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_05]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_05]
[0m[0m16:04:12,667 INFO [com.twitter.hbc.httpclient.ClientBase] (hosebird-client-io-thread-0) tweetStream-client exit event - Connection pool shut down
[0m[33m16:04:12,667 WARN [com.twitter.hbc.httpclient.ClientBase] (hosebird-client-io-thread-0) tweetStream-client failed to establish connection properly
[0m[0m16:04:12,667 INFO [com.twitter.hbc.httpclient.ClientBase] (hosebird-client-io-thread-0) tweetStream-client Done processing, preparing to close connection
[0m[0m16:04:12,667 INFO [com.twitter.hbc.httpclient.ClientBase] (hosebird-client-io-thread-0) tweetStream-client Shutting down httpclient connection manager
I tried to identify why this error occurs when I do this and find https://dev.twitter.com/docs/streaming-apis/connecting which says
The client has connected too frequently. For example, an endpoint returns this status if:
A client makes too many login attempts in a short period of time.
Too many copies of an application attempt to authenticate with the same credentials.
My question is simple:
With one registered application and using streaming-api, how can I have separate queues for different items to track without connecting to the client every time. This is so that my application works maintains independent queues for each track-to-follow and without hitting Rate Limits
Thanks a lot for your help in advance
