Search API returning (very) sparse geocode results

[jimboWelsh]

Sure thing, try using a placeID + keywords as your query rather than coordinates and a radius. For example, looking for original tweets in the USA containing the words ‘foo’ or ‘bar’:

query = ‘place:96683cc9126741d1 “foo” OR “bar” -“rt @”’

query command = aTweepyAPIObject.search(q = query,
since_id = lastTweetID,
result_type=“recent”,
count = 100)

This will get you a high percentage of recent tweets in the USA or any other country, then you just need to find the right frequency and find a way to pare down the volume to the sub location you want.

[jimboWelsh]

Everything about the tweet geolocations is always a bit fuzzy, so I really don’t feel that using one coordinate method or another is an end all solution. If you check the tweet location data, you’ll find that some if not most of the locatable tweets for a given coord+radius search conspicuously appear to be from somewhere else. For example, I’ve found tweets ostensibly from the US east coast that had a location name and coordinates from other countries with language and events described matching local events in said country. Going with the highest volume of proximal tweets and then narrowing it by location seems the best approach.

[nicktaylor3]

I’ve been using your method all along and it doesn’t work for me any better than any other method. I still only see posts that are explicitly geotagged, and not tweets where the location is inferred from the profile.

[avatter]

Q2 you guys wanted to get started; I guess that there are no good news yet?

[SocialBiz]

Ryan, can you or someone from Twitter please update the developer community on the status of this issue. We are approaching the one year anniversary of this thread and despite several indications that Twitter is committed to fix this issue there has been no progress or update. In fact, there has been dead silence for over 3 months now.

As a matter of professional courtesy to your developer community, could you please respond and let us know if there is any hope or expectation that this issue will be addressed and fixed anytime soon.

Thanks.

[nicktaylor3]

If anything, this problem is getting worse. In addition to not returning tweets based on profile data, my tests are now returning only a fraction of properly geotagged tweets as well.

[lampos]

Any news on this problem? Have you finally made the decision to turn this useful feature off?

[GoChatterUP]

An update would be greatly appreciated.

Thank you.

[mikekotsch]

That’s a real bummer.

[sohussain]

I’m adding my comment here to let twitter Devs know that we still need a solution to this problem.

[LukeInTH]

#helloworld can we get an update. Its been almost a year since this stopped working.

[lampos]

+1 !!

[nicktaylor3]

+2

[andypiper]

Sorry for the prolonged lack of communication while we’ve been looking at how best to address this.

Around the same time that the search issue occurred, we were also in the process of making some changes to Twitter’s apps which modified the ways in which users were encouraged to share their location information - the move to places, rather than specific geocoordinates. As part of this change, the ability to add a precise set of coordinates to a Tweet is not as visible as it was previously.

The percentage of Tweets tagged with geolocation data by users who choose to share that information has historically been low: ~2% of Tweets have been tagged. The change across to place data means that the number which have exact coordinates is very small. We’re committed to respecting our users’ privacy, so we don’t surface location information they have not chosen to share.

So, the ongoing lower volumes of Tweets seen following this change also flows into some of the UI and UX changes in the Twitter app; and ultimately, will continue to mean that geo search results on the public API remain low volume.

We do offer an enriched geolocation search via the enterprise Gnip APIs, which are fully supported and can provide some additional fallback defaults in cases where a user has tagged his or her location in their profile, but not on the Tweet. This may be an option for some developer use cases, although it is a commercial option. At our recent Flight conference, we shared some more information on how this works, and a video replay of that session will be online very soon.

Finally, we’re aware of the appetite for an enhanced search API and we are investigating whether this is something we can provide as part of the recent #HelloWorld feedback. Stay tuned for more on this.

[johnbarratt]

Hi Andy,
Thanks for the detailed update. However the problem here wasn’t related at all to the places change, which have reduced the precisely located (GPS) tweets available as a proportion of explicit ‘geo’ tweets. The problem here is due to the geocoded public ‘location’ string of a user not being used in the Search API to locate tweets as it was. This functionality is currently available via the public web search, it’s just not also being used in the API.

Thanks again,

JB

[nicktaylor3]

I have to agree with John that this answer doesn’t really seem to address the issue at hand. The problem we’ve been having affects both co-ordinate/radius searches and searches using Places.

I can appreciate that there a privacy issues around location, but what we’re trying to do doesn’t use information that the users have chosen not to share – it’s shared on the their profile. As John says, we can search by this information on the Twitter website, so it’s unclear why it’s a privacy issue for the API.

Is the bottom line simply that this feature is being removed from the API and only included in the paid offerings? If that’s the case I’m sure everyone in the thread would appreciate that clarity.

[andypiper]

Apologies for not being clear enough.

The profile fallback was disabled due to an issue late last year, and is not something that we have plans to restore in the public API at the moment, as a) doing so is not a trivial effort and b) other development efforts have taken priority.

As I mentioned, we are also evaluating the many #HelloWorld suggestions and how we could potentially provide an enhanced public search API (and a number of other things), but we can’t say when those changes might happen.

The Gnip API continues to support several enriched search options.

[jkgeyti]

Hi Andy,

I’m sure we’re many who haven’t been happy with the way communication about this issue has been handled during the last year+ it’s been going on (development must be a mess behind the scenes?), but thank you for finally settling on an answer, even though it’s a WONTFIX. At least we can now explore other avenues, knowing geocoding won’t suddenly be reinstated in the near future.

[nicktaylor3]

Thanks for the clarity, Andy. It would be great if the withdrawal of this feature could be reflected in the API documentation, specifically on these pages:

https://dev.twitter.com/rest/reference/get/search/tweets
https://dev.twitter.com/rest/public/search

[nsdesign]

While the response is appreciated… it’s still madness…

Do this search on Twitter search (at the top of the Twitter website):
walk OR walking AND dog geocode:55.86,-4.25,50mi

And you’ll get loads of results from people in the Glasgow area tweeting about walking their dog!

DO the same search on something like Hootsuite, and you’ll be lucky to get just 1 or 2 results (ie: the people who have turned on location specifically).

Surely you can allow this to fully function on apps, and 3rd partys like Hootsuite just as it used to do over a year ago!!