#1

Hi,
My goal is to pull recent tweets and their expanded media objects using the tweets/search/recent endpoint.

Using
endpoint: tweets/search/recent
query params:
'query': {'from:attn'}
'tweet.fields': 'attachments,text'
'media.fields': 'duration_ms,height,media_key,preview_image_url,type,url,width,public_metrics'
'expansions': 'attachments.media_keys'

… only the tweet fields and media_ids are returned. All of the media fields requests are ignored and not returned.

Using
endpoint: tweets
query params:

'ids': [1322199080148361219]
'media.fields': 'duration_ms,height,media_key,preview_image_url,type,url,width,public_metrics'
'expansions': 'attachments.media_keys'

All media fields are appropriately returned for this one tweet.

Why are media expansions not working on the recent tweets endpoint?

1 Like
#2

Thanks for reaching out here. Do you have the full version of the request you are calling? After I get this from you I’ll look into why this might be happening.

#3

Hi Jessica, thanks for the reply.

This was either quietly fixed or I was doing something wrong last week.

Here is a working query for others (and my future self):

curl "https://api.twitter.com/2/tweets/search/recent?query=from:attn&expansions=attachments.media_keys&media.fields=duration_ms,height,media_key,preview_image_url,type,url,width,public_metrics&max_results=20" -H "Authorization: Bearer <redacted>"

Gets me the expanded results I expect with media metrics under the includes key.

When I add 'public_metrics','non_public_metrics','organic_metrics' I have to use a python library to implement oAuth1.0a.

2 Likes
#4

I’m super green to APIs. I’ve read over the documentation and found an example there and your example but all the media fields are still missing for me (except the media_key). Any chance you can test this? (I’m using the API v2 Postman collection. Thank you in advance!

https://api.twitter.com/2/tweets/search/recent?max_results=100&query=%23dune -is:retweet&tweet.fields=public_metrics,created_at,author_id&expansions=attachments.media_keys&media.fields=duration_ms,height,media_key,preview_image_url,public_metrics,type,url,width

#5

@joshpresto The media will be in the includes part of the response, not attached to the tweets in data directly, but referenced by media_key

#6

Thank you Igor! Found the media details now in the response… I tried to flatten the json output into a csv file using json-csv.com as well as Tableau, but neither can pair the media to the correct tweet. In Tableau’s case the “id” isn’t referenced in the same schema level as “media”. (see screenshot). Any ideas on how I might use the twitter api responses to pair the id with the media? (I’m not a coder, so be gentle:))

image
image1626×1664 238 KB

#7

@joshpresto Sure - this is exactly the use case for twarc:

(also for reference: while the focus of twarc is for Academic use, it will work just as well with Standard Access v2 API, the only thing that won’t work is --archive when using twarc2 search and some search operators, in case anyone else is wondering)

After you’ve gathered a dataset with twarc somehow, you can use

twarc2 flatten mydata.json mydata_flat.json

to “flatten” the representation. Or add --flatten to the `twarc2 search command as an option…

twarc2 search --flatten "query" output.json

Also, you can use: GitHub - DocNow/twarc-csv: A plugin for twarc2 for converting tweet JSON into DataFrames and exporting to CSV. this to convert to a flat CSV representation (doesn’t matter in what format, as long as it’s v2 it will figure it out).

1 Like
#8

Thank you Igor for the resource recommendation AND the step-by-step setup. I’m going to cut my teeth on this and try to solve for my specific use case. thank you thank you!

1 Like