Streaming data into a data store

foundashoe · 2013-08-27T03:35:56.000Z

Hi

Firstly, i believe i am about to endevour into something new for me, so i thank you for any advice but also apologise for being a noob. I will try and choose my words carefully. I am having a meeting on this tomorrow so am afraid i do not have the upfront reading time needed to find previous posts, however if this has been answered a lot before, please simply tell me and i will endevour to find those posts. If there is an article on this, kindly forward me to it.

I believe what i need to build is a streaming client, as opposed to REST or Search. I would like to access specific tweets in relation to key words. For example to find all tweets related to an earthquake. The tweets should preferably come from ground zero (or close to) of the EQ, and not from people sending out heart felt sympathys from across the globe.

Question 1. WIll i need to built the streaming client from scratch? Or does Twitter offer these streaming clients.
Question 2. Is there any material on how to build this? I would like to customize some sort of alghorithm that does not store the junk in our data store. I would want for example tweets about a flooded city, not a kid complaining about being flooded with homework.

Thanks in advance

foundashoe · 2013-08-30T00:32:03.000Z

Bump. Although my question is now more to th epart of building something to use the Streaming API.

Can anyone tell me if I need to find someone who can build me something from scratch, or is there something avaiklable we can modify to store tweets using the streaming API?

LogicalArthur · 2013-08-31T00:38:08.000Z

Hi Aaron:

Thanks for your question–it’s prompted us to update our Twitter Libraries listing!

In particular, you might focus your attention here (if you’re writing Java): https://github.com/twitter/hbc

We also offer links to third-party libraries in other languages that you can try out, so you shouldn’t have to build the API interaction piece from scratch (but it will be important that you understand on a high level how authentication and API requests work).

That having been said, any kind of robust filtering would have to be done in your own code. However, you may want to take a look at the following documentation to help you out: track parameter

Since you mention storing Tweets, please make sure you’re familiar with the limitations on how you are allowed to do that by reviewing our developer rules of the road (especially section I.).

Good luck!

foundashoe · 2013-09-04T04:48:14.000Z

Great, many thanks for the feedback and a pathway forward. Will use this as basis to move forward in application.

Again, thank you.

LogicalArthur · 2013-09-04T19:53:58.000Z

You’re very welcome!