I’ve been using the searchtweets-v2 python package and it works great - thanks @IgorBrigadir for putting me onto it and for making your ‘carp’ tutorial in GitHub; it made things so much easier. However I’m finding that the output changes format between requests/paginations - see the screenshot where the tweet fields (‘text’, ‘author_id’, ‘created_at’, etc.) start being shown in a different order (at first it is ‘text’, followed by the others, and then it changes). This happens every ~500 lines because of the request cap. Any suggestions? Thanks a lot in advance! twitterjson|690x489
2 Likes
Haha! I haven’t finished the carp thing ( GitHub - igorbrigadir/carp: 🐟 Twitter Carp Data ) so it’s missing a bunch of notes and instructions…
The json will sometimes change around - this is expected, sometimes fields in json are arranged differently and any json parser will be able to handle that.
For human readability it can be annoying but generally i prefer to use jq jq to “pretty print” it if i need to manually examine json.
Another thing to watch out for in search-tweets-v2 is the output format - by default it outputs tweets, and then separately outputs includes and expansions. How do you work with the output from this search client? · Discussion #116 · twitterdev/search-tweets-python · GitHub
it’s not merged yet https://github.com/twitterdev/search-tweets-python/pull/112 but if you want to try it:
you can
pip uninstall searchtweets-v2
and then
pip install git+https://github.com/twitterdev/search-tweets-python@refs/pull/112/merge
and then run it with --atomic command line switch. like
search_tweets.py --atomic ...
to get 1 tweet per line, that will include all the expansions inline.
1 Like
That’s great - thanks so much for your help! Again!
