Streaming tweets that stem from a tweet from a certain user

[dshandler]

Hi,

I was wondering if there was a method free (or paid) that can stream tweets that have replied to a specific user or is a follow on tweet from the same specific user.

For example, just say the specific user is @realdonaldtrump. I would like to access all the tweets that have followed on from a tweet from @realdonaldtrump.

My aim is to analyse conversation lengths of a specific user rather than a topic (which is how I stream at the moment).

I am able to just sort the data that I stream from entering the search term realdonaldtrump, such that I only use data which stems from the user @realdonaldtrump but I would like to maximise the data points that I can capture through streaming because a lot of tweets replying to @realdonaldtrump won’t have that term in the tweet.

Thanks

[andypiper]

You could use the is:reply operator in the enterprise PowerTrack API. This is not available in the standard streaming API.

[dshandler]

Thanks Andy.

That is kind of what I’m looking for, however is that referencing the in_reply_to_user_id? If it is, it would only find the first tweet that replied to @realdonaldtrump and all my conversations would be of length 2. Or does this reference “people in conversation” (see pics) that is visible when browsing twitter through the browser or app. From what I could see this information isn’t available in a single tweet and is generated, but I could be wrong and will be happily corrected.

[Aurelia]

Hi @dshandler - the Tweet payload returns an in_reply_to_status_id field. Taking your example, you could pass a query that would return all Tweets that respond directly to the first Tweet of the thread (the original Tweet). You would get a series of Tweets back, all of which are a direct response to the original Tweet. Example query below (using Historical PowerTrack).

So to answer your first question: with this example query, the number of Tweets returned would be a lot more than 2.

{
    "value":"in_reply_to_status_id:1070110927788347393",
    "tag":"test_1"
 }

However, this example request would not pick up on “sub-conversations”, such as the Tweet you screenshotted.

I’ve pasted below the first part of the Tweet payload that corresponds to the Tweet in your screenshot. You can see that the in_reply_to_status_id field references a different status id: this is the status id that the user was directly responding to here (and not the status id that first started the thread).

If you wanted to know all the “people in conversation”, you can use the user_mentions field to see all people who are included in this response (including the very first user who started the thread).

{
  "created_at": "Wed Dec 05 00:33:51 +0000 2018",
  "id": 1070113988245999616,
  "id_str": "1070113988245999616",
  "full_text": "@jstrong244 @krassenstein @EdKrassen @realDonaldTrump Cant handle the truth mate?",
  "truncated": false,
  "display_text_range": [
    54,
    81
  ],
  "entities": {
    "hashtags": [],
    "symbols": [],
    "user_mentions": [
      {
        "screen_name": "jstrong244",
        "name": "Jasaplay",
        "id": 2868067125,
        "id_str": "2868067125",
        "indices": [
          0,
          11
        ]
      },
      {
        "screen_name": "krassenstein",
        "name": "Brian Krassenstein",
        "id": 133938408,
        "id_str": "133938408",
        "indices": [
          12,
          25
        ]
      },
      {
        "screen_name": "EdKrassen",
        "name": "Ed Krassenstein",
        "id": 132339474,
        "id_str": "132339474",
        "indices": [
          26,
          36
        ]
      },
      {
        "screen_name": "realDonaldTrump",
        "name": "Donald J. Trump",
        "id": 25073877,
        "id_str": "25073877",
        "indices": [
          37,
          53
        ]
      }
    ],
    "urls": []
  },
  "source": "<a href=\"http:\/\/twitter.com\" rel=\"nofollow\">Twitter Web Client<\/a>",
  "in_reply_to_status_id": 1070113599199371265,
  "in_reply_to_status_id_str": "1070113599199371265",
  "in_reply_to_user_id": 2868067125,
  "in_reply_to_user_id_str": "2868067125",
  "in_reply_to_screen_name": "jstrong244",
  "user": {
    "id": 830612821155082241,
    "id_str": "830612821155082241",
    "name": "Vince Smith",
    "screen_name": "TheNutter64",
    "location": "Brisbane, Queensland",
    "description": "",
    "url": null,
    "entities": {
      "description": {
        "urls": []
      }
    },
ETC.

[dshandler]

Hi @Aurelia

Thanks! I’m not specifically looking for certain tweets though, that was just an example that I looked up. User mentions is what I’m looking for, so thank you! And just an aside, It would be helpful if tweets included a status_id field within user mentions to locate the exact status that was replied to or null if it really is just a mention. I thought it was specifically if someone used an @ in their tweet but great to know it includes replies.

I’m specifically trying to capture conversations occurring on twitter, so I mean you would receive many many tweets but when they are combined into conversations it would just be a single tweet with a reply and you’d have a conversation length of 2, nothing longer.

At the moment I sort the tweets into conversations after streaming them into a file using the “in_reply_to_status_id”, however I get a lot of singletons and retweets which are useless to me. At the moment I end up discarding 60-80% of tweets I stream because I am not interested in retweets and then of the remaining 20-40% of tweets that are original, only a quarter end up forming part of a conversation. So, for one of my streams,16 million tweets becomes just under a million when sorted into conversations. I’d really like to limit my stream to maximise the number of conversations I can save.

I think I’ve found what I was looking for: the rules and filtering page. So according to what you’ve said I can query with @NASA and it will return all tweets that have NASA “in the conversation” or directly mention @NASA in their tweet? And the “@” query should include tweets that were found using the “to:” query from that page because all replies go into the user mentions field? And if I wanted to get a larger volume of tweets I’d need to pay for the enterprise access and filter realtime tweets using the PowerTrack API? Is there a price range for this access?

There is also typos on the page that @andypiper linked to. Specifically the description for the is:reply operator where it says: “You can use is:reply in conjuction with is:rewteet and is:quote to only deliver replies to original Tweets.”

[hmz]

It sounds like you want to track an entire reply tree of a Tweet, which is not easy to do with the products that we currently have available.

Based on your last response, it sounds like might already be enterprise customers of ours, considering you used the in_reply_to_status_id operator. If you are, you can filter out retweets by negating the is:retweet operator.

The best thing that you can do is build in logic to continuously build in_reply_to_status_id rules that include replies from the original Tweet in question. For example, if original Tweet A has 3 replies, then create an in_reply_to_status_id for not only A, but also for the 3 replies so you can also get the replies for those Tweets.

Let me know if this makes sense.

[dshandler]

Thanks @hmz

Yeah that’s correct, I would like to keep track of the reply tree which involve a certain user at any point in the conversation. At the moment I use that operator and that logic locally on files of tweets I have saved and I see I will need enterprise access to filter prior to saving tweets.

I don’t actually have access to enterprise yet and I will be considering it at some point, in maybe a couple months when everything is finally developed. Can you give me pricing info if I want access to the Enterprise PowerTrack? Or how do I go about getting a quote?

[Aurelia]

Hi @dshandler - please fill out this form if you’re interested in enterprise access: https://developer.twitter.com/en/enterprise-application Someone from the sales team will then get back to you with approximate subscription costs.