#1

Hi everyone,

I am new with API Twitter v2 and i try to get conversations data, but i have not good results. My objective is: comments from users, sub-comments, etc. The next figure represent my goal (https://twitter.com/nytimeses/status/1173984627427557376):

ex_2
ex_2889×536 207 KB

My code for the example is:

def make_request_comments(headers,id_conversation):
    url = "https://api.twitter.com/2/tweets/search/recent"

    query = '(conversation_id:'+str(id_conversation)+')'
    query_params = {'query': query,
                    'expansions': 'geo.place_id,author_id',
                    'tweet.fields': 'id,text,author_id,in_reply_to_user_id,conversation_id,created_at',
                   }   
    return requests.request("GET", url, params=query_params, headers=headers).json()

id_conversation = 1173984627427557376
response = make_request_comments(headers,id_conversation)
response['data']

… und my results are:

[{'author_id': '1406381522073145344',
  'in_reply_to_user_id': '2975678883',
  'conversation_id': '1173984627427557376',
  'id': '1410791239176396801',
  'text': '@nytimeses @JorgeGCastaneda Fraude electoral del @JNE_Peru #AuditoriaElectoral https://t.co/waViPMRmjx',
  'created_at': '2021-07-02T02:43:37.000Z'},
 {'author_id': '1402745602161643520',
  'in_reply_to_user_id': '2975678883',
  'conversation_id': '1173984627427557376',
  'id': '1409712142824001537',
  'text': '@nytimeses @JorgeGCastaneda WE NEED HELP FROM YOU THE ELECTIONS IN PERU ARE FRAUDULENT, WE HAVE PROOF. THE CURRENT GOVERNMENT IS CORRUPT, WE DO NOT WANT TO BE COMMUNISTS, WITH AN AUDIT THE TRUTH WILL BE KNOWN. AYUDEN A PERU,  COMUNISTAS NUNCA,HAY FRAUDE QUEREMOS  AUDITORIA INTERNACIONAL',
  'created_at': '2021-06-29T03:15:40.000Z'},
 {'author_id': '877348165032099840',
  'in_reply_to_user_id': '2975678883',
  'conversation_id': '1173984627427557376',
  'id': '1409702999304269825',
  'text': '@nytimeses @JorgeGCastaneda MÉXICO el único País del mundo donde los niños con cáncer son catalogados como un peligro para dar  un golpe de estado.\n#GatellAsesino\n#NinosConCancer \n#NinosGolpistas https://t.co/rQp4VqIA6G',
  'created_at': '2021-06-29T02:39:20.000Z'},
 {'author_id': '3297909826',
  'in_reply_to_user_id': '2975678883',
  'conversation_id': '1173984627427557376',
  'id': '1409154186734931970',
  'text': '@nytimeses @JorgeGCastaneda Mientras haya pandemia por combatir, todos los países están al nivel de la pobreza extrema, sobre todo en materia sanitaria.',
  'created_at': '2021-06-27T14:18:33.000Z'}]

I would greatly appreciate your help, I have the following complications:

  • I need extract all comments, same in previous picture, in total are 597… I don’t understand, why i have only 4 results? :grimacing:
  • I cant extract comments from comments, for built all thread from conversation… what is the best form for this?. The next picture show my objective, however, it could be that that last comment has another associated… the idea is capture all comments relationed.:
    ex_3

I appreciate very much the comments,

Greetings
C.

#2

This will only search within the last 7 days, and your example tweets are from years ago - to get all you would need access to the “Academic Access” /search/all endpoint.

#3

Thank you very much @IgorBrigadir, I have put a bad example. Updating a recent news item, within 7 days I get 10 comments … is it a limitation of a basic version?

(https://twitter.com/nytimes/status/1411404401357512708)

def make_request_comments(headers,id_conversation):
    url = "https://api.twitter.com/2/tweets/search/recent"

    query = '(conversation_id:'+str(id_conversation)+')'
    query_params = {'query': query,
                    'expansions': 'geo.place_id,author_id',
                    'tweet.fields': 'id,text,author_id,in_reply_to_user_id,conversation_id,created_at',
                   }   
    return requests.request("GET", url, params=query_params, headers=headers).json()

id_conversation = 1411404401357512708
response = make_request_comments(headers,id_conversation)
len(response['data'])

The result is 10 comments.

Greetings
C.

#4

Ah ok, for a more recent tweet it should work - you may have to specify max_results too.

I get 55 tweets for that.

I’m using twarc2: twarc2 (en) - twarc

twarc2 conversation 1411404401357512708 output.jsonl

which calls

https://api.twitter.com/2/tweets/search/recent?expansions=author_id%2Cin_reply_to_user_id%2Creferenced_tweets.id%2Creferenced_tweets.id.author_id%2Centities.mentions.username%2Cattachments.poll_ids%2Cattachments.media_keys%2Cgeo.place_id&user.fields=created_at%2Cdescription%2Centities%2Cid%2Clocation%2Cname%2Cpinned_tweet_id%2Cprofile_image_url%2Cprotected%2Cpublic_metrics%2Curl%2Cusername%2Cverified%2Cwithheld&tweet.fields=attachments%2Cauthor_id%2Ccontext_annotations%2Cconversation_id%2Ccreated_at%2Centities%2Cgeo%2Cid%2Cin_reply_to_user_id%2Clang%2Cpublic_metrics%2Ctext%2Cpossibly_sensitive%2Creferenced_tweets%2Creply_settings%2Csource%2Cwithheld&media.fields=duration_ms%2Cheight%2Cmedia_key%2Cpreview_image_url%2Ctype%2Curl%2Cwidth%2Cpublic_metrics&poll.fields=duration_minutes%2Cend_datetime%2Cid%2Coptions%2Cvoting_status&place.fields=contained_within%2Ccountry%2Ccountry_code%2Cfull_name%2Cgeo%2Cid%2Cname%2Cplace_type&query=conversation_id%3A1411404401357512708&max_results=100

(and if there are more tweets, you will have to paginate - twarc can also do this for you, which i recommend instead of writing your own code)