REST API returns malformed JSON

M0_84 · 2016-07-27T15:42:56.978Z

I’m trying to get the list of someone’s followers.
But REST API sometimes returns malformed JSON which cannot be parsed as usual, so my app stops every time even I decrease the “count” parameter.
This is a REAL returned data from REST API which crashes my app.
https://gist.github.com/nao20010128nao/9922801226373270b1b419e5e048df0d

andypiper · 2016-07-27T17:22:17.277Z

Are you able to provide the API call and parameters that produces that response, please?

What are you using to parse the JSON inside your app?

M0_84 · 2016-07-28T10:59:03.069Z

API call is this (this is not a call when I posted, but it also returns malformed JSON):

GET followers/list?user_id=783214&count=50&cursor=1541075414351958046

I can’t dump the response because Wireshark doesn’t work well.

The library to parse the JSON is Newtonsoft.Json (used by CoreTweet).

andypiper · 2016-07-28T13:11:14.710Z

I’ve just tried this using twurl and the response from that call seems to parse OK in JSONlint. I don’t see anything that breaks at the moment.

I can see that there is a malformed element in the raw.json file in your first post. However, I’m confused as to how you generated that JSON document - what tool or app are you using? If this is your own app, can you share the app ID (not the consumer keys), please?

Basically I’m struggling to reproduce this reliably.

andypiper · 2016-07-28T13:13:36.540Z

Also, in your original post, the Tweet that appears to have “malformed” content which breaks the parser appears to be Tweet ID 758307575939133440 (please confirm that this matches your analysis?) - I can successfully pull that individual Tweet from the API and do not see this issue.

M0_84 · 2016-07-28T13:52:29.815Z

Umm…

This is the code of the request of CoreTweet(used by my app)

github.com

CoreTweet/CoreTweet/blob/master/CoreTweet.Shared/Internal/TokensBase.cs#L448


            return type == MethodType.Post
                ? Request.HttpPost(uri, prmArray, CreateAuthorizationHeader(type, uri, prmArray), options)
                : Request.HttpNoBody(type, uri, CreateAuthorizationHeader(type, uri, null), options);
        }
        catch(WebException ex)
        {
            var tex = TwitterException.Create(ex);
            if(tex != null) throw tex;
            throw;
        }
    }
#endif
}
}

github.com

CoreTweet/CoreTweet/blob/master/CoreTweet.Shared/Internal/Request.cs#L68


        x.WriteTo(stream);
        stream.WriteString("\r\n");
    }
    stream.WriteString("--" + boundary + "--");
}
#endif


internal const DecompressionMethods CompressionType = DecompressionMethods.GZip | DecompressionMethods.Deflate;


#if SYNC
internal static HttpWebResponse HttpNoBody(MethodType type, Uri url, string authorizationHeader, ConnectionOptions options)
{
    if(options == null) options = ConnectionOptions.Default;


    string method;
    switch (type)
    {
        case MethodType.Get: method = "GET"; break;
        case MethodType.Post: method = "POST"; break;
        case MethodType.Delete: method = "DELETE"; break;
        default: throw new ArgumentOutOfRangeException(nameof(type));

And this is the app ID of my app: 12278991

andypiper · 2016-07-28T15:21:50.104Z

I’m still unable to reproduce this issue with a specific Tweet or API response that contains a malformed element. I accept that the JSON you’ve posted at the top of this thread is not valid, but I cannot recreate it.