Obtaining specific data of a tweet


#1

I need to develop an application to obtain the following data from a sibgle tweet or hashtag:

  1. post
  2. users
  3. reach
  4. impressions
  5. related topics
  6. top posts
  7. top sites
  8. location
  9. share of post
  10. sentiment
  11. demographics

Im new using the api, please help me


#2

Hi. You’re going to want to become friends with https://dev.twitter.com/rest/tools/console

You’ll also need these:
https://dev.twitter.com/rest/public
https://dev.twitter.com/streaming/overview

What you’re looking to do is not a simple process. From what I’ve read, some of what you’re looking for won’t be available - unless you use Gnip.

A good place to start would be to look at how other libraries communicate with Twitter.
https://dev.twitter.com/overview/api/twitter-libraries


#3

Thank you for responding, I am currently connecting laravel with Abraham / TwitteroAuth library using the search api. But I need more data


#4

Unfortunately I’m not familiar with Laravel, but I think I’ll look into it for another project. You’ll want to use both the Stream and REST APIs. I generally use Stream to see what’s going on right now, and REST to get granular.

Both of these may help.


Now on the point to point

  1. When using search API, you’ll get the tweet (post) do check mark.
  2. Search can give you up to 7 days back of users who have tweeted it, but I believe it’ll just be a sample. You’ll need to use Stream to collect full activity.
  3. That’ll be interesting. You’ll need to to use REST to get followers count of every user who has tweeted whatever you’re looking for.
  4. Twitter has stated this is not a priority for them, unless you go with Gnip.
    REST API support for Impressions data?
  5. Look for other hashtags the users have included in the tweets you’ve collected. For example #ios would also include #apple, #iphone, etc. Strip the tweet into an array of words, collect all words starting with #, then do a frequency analysis. Chances are the top 5 are most relevant. I say chances are because depending on your specific search, you may get more noise than quality.
  6. retweet_count and favorite_count will get you there.
    https://dev.twitter.com/overview/api/tweets
  7. Urls_entity is what you’re looking for
    https://dev.twitter.com/overview/api/entities-in-twitter-objects
  8. Coordinates is what you’re looking for. Location may not be relevant. Refer to 6 for link.
  9. I don’t understand.
  10. That’s somewhat difficult. You will need to find a tool that uses natural language processing to help you with that. There are some great Python tools.
    http://www.nltk.org
    Or read these and roll your own.
    https://en.m.wikipedia.org/wiki/Sentiment_analysis
    https://blog.hootsuite.com/beginners-guide-sentiment/
  11. That seems to only be available for ad campaigns. Going deeper into what I said in 10, you could make somewhat educated guesses about the demographics about the users if you had enough tweets from each user. But there are a lot of variables. Best to research this further. There may be services, libraries, or techniques out there already.

All this being said:
First you need to fully understand what you want. Whip out your notepad and write exactly what you want, step by step.
Second you need to understand what Twitter can/will provide. Reading and then rereading all the documentation is a must.
Finally you’ll understand what you can do.

There are plenty of tools, tutorials, libraries to help you. Don’t try to recreate the wheel, unless you’re that kind of person. I know I am, so I know it’s usually not worth the effort, unless the goal is learning itself.
(Disclaimer: when using other people’s tools or libraries always be sure to follow their licenses.)


#5

Thank you very much for taking the time to read and answer my questions, I’ll take you step by step council.