Language in some results not in ISO-639-1 standard

aldnet · 2016-10-18T15:57:14.916Z

He are receiving some tweets with a language not prsent in ISO 639-1 Tables

In this page https://dev.twitter.com/rest/reference/get/search/tweets is indicated that lang par is from ISO 639-1, some we assume that the results comes from the same standard.

http://www.loc.gov/standards/iso639-2/php/code_list.php

One example is the following tweet

https://api.twitter.com/1.1/statuses/show/788010531525431296.json

Where the language is “in”, which language is that?

We have around of 4k tweets from an scenario of 300k with that language.

aldnet · 2016-10-19T14:40:18.357Z

I’m assume that the problem is the same of this post:

Hebrew and Indonesian getting incorrect 'lang' codes

Hello. Any news about this topic? I also met problem with Indonesian. The problems also occurs when filtering stream by the language. Simply filtering by 'id' (indonesian real code) returns no results Regards