Http Twitter Stream - not found or forbidden


#1

trying to connect to Twitter stream with http. trying every URL I can find. Please see my code and errors. Here ismy Post content:
authHeader : "OAuth oauth_nonce=“NjM1NTU4MjA3NTEwNTA5MzMx”, oauth_signature_method=“HMAC-SHA1”, oauth_timestamp=“1420241951”, oauth_consumer_key=“F1if6ZvPaWcbrhq81leLFNOhT”, oauth_token=“859829466-AdZSBtACvpbQlY1hGgGi0YCVJJG0LgzhEP3HkWoK”,

oauth_signature=“A7XUQV3GtRHKpTL1%2BYE5AqBGwzs%3D”, oauth_version=“1.0"” String
postBody “status=Hoping%20to%20be%20authorized” String

------------------------------ http code follows: ---------------------------------------

        ' oauth implementation details
        Dim oauth_version = "1.0"
        Dim oauth_signature_method = "HMAC-SHA1"

        ' unique request details
        Dim oauth_nonce = Convert.ToBase64String((New ASCIIEncoding()).GetBytes(Date.Now.Ticks.ToString()))
        Dim timeSpan = Date.UtcNow - New Date(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc)
        Dim oauth_timestamp = Convert.ToInt64(timeSpan.TotalSeconds).ToString()

        ' message api details
        Dim status = "Hoping to be authorized"
        Dim resource_url = 
        "https://api.twitter.com/oauth/request_token" not found
        '"https://api.twitter.com/oauth/access_token" not found
        '"https://api.twitter.com/1.1/" not found
        '"https://stream.twitter.com/1.1/" not found
        '"https://api.twitter.com/oauth/authorize" forbidden
        '"https://stream.twitter.com/1.1/" forbidden
        '"https://stream.twitter.com/1.1/statuses/filter.json"  forbidden

        ' create oauth signature
        Dim baseFormat = "oauth_consumer_key={0}&oauth_nonce={1}&oauth_signature_method={2}" & "&oauth_timestamp={3}&oauth_token={4}&oauth_version={5}&status={6}"
        Dim baseString = String.Format(baseFormat, oauth_consumer_key, oauth_nonce, oauth_signature_method, oauth_timestamp, oauth_token, oauth_version, Uri.EscapeDataString(status))

        '>>> baseString = here 

        baseString = String.Concat("POST&", Uri.EscapeDataString(resource_url), "&", Uri.EscapeDataString(baseString))
        Dim compositeKey = String.Concat(Uri.EscapeDataString(oauth_consumer_secret), "&", Uri.EscapeDataString(oauth_token_secret))

        'endcode the signature
        Dim oauth_signature As String
        Using hasher As New HMACSHA1(ASCIIEncoding.ASCII.GetBytes(compositeKey))
            oauth_signature = Convert.ToBase64String(hasher.ComputeHash(ASCIIEncoding.ASCII.GetBytes(baseString)))
        End Using

        ' create the request header
        Dim headerFormat = "OAuth oauth_nonce=""{0}"", oauth_signature_method=""{1}"", " & "oauth_timestamp=""{2}"", oauth_consumer_key=""{3}"", " & "oauth_token=""{4}"", oauth_signature=""{5}"", " & "oauth_version=""{6}"""
        Dim authHeader = String.Format(headerFormat, Uri.EscapeDataString(oauth_nonce), Uri.EscapeDataString(oauth_signature_method), Uri.EscapeDataString(oauth_timestamp), Uri.EscapeDataString(oauth_consumer_key), Uri.EscapeDataString(oauth_token), Uri.EscapeDataString(oauth_signature), Uri.EscapeDataString(oauth_version))
        ' make the request aka "command"
        Dim postBody = "status=" & Uri.EscapeDataString(status)
        ServicePointManager.Expect100Continue = False

        '------------------------------- FIRST TRY
        Dim request As HttpWebRequest = CType(WebRequest.Create(resource_url), HttpWebRequest)
        request.Headers.Add("Authorization", authHeader)
        request.Method = "POST"
        request.ContentType = "application/x-www-form-urlencoded"
        Dim content() As Byte = ASCIIEncoding.ASCII.GetBytes(postBody)

        Using stream As Stream = request.GetRequestStream()
            'writes a sequence of bytes to the current stream, and advances the position to content.length the end of the stream
            stream.Write(content, 0, content.Length)
        End Using

        'after sending Post what is the response?

        Dim response As WebResponse = request.GetResponse()  '<<<<<<<<<,The remote server returned an error: (401) Unauthorized.

        ''=============================== SECOND TRY
        'Dim req As WebRequest
        'Dim res As HttpWebResponse

        'req = WebRequest.Create(resource_url)
        'req.Timeout = -1
        'req.Headers.Add("Authorization", authHeader)

        'res = DirectCast(req.GetResponse(), HttpWebResponse)
        'Dim streamReader As StreamReader
        'Dim strLine As String
        'Dim encode As Encoding = System.Text.Encoding.GetEncoding("utf-8")

        'streamReader = New StreamReader(res.GetResponseStream(), encode)
        'While True
        '    strLine = streamReader.ReadLine()
        '    MsgBox(strLine) 'temp just in case it ever works!
        'End While

        '====================================

#2

Are you using ASP.NET here? I’m not familiar with the language implementation I’m looking at, sorry :confused:

Is there a reason not to use an existing .NET library that handles Twitter OAuth for you? we have a few listed and it would save you having to do some of the heavy lifting yourself.


#3

Andy - Thanks for you time, I appreciate it. re: reason not using existing .Net .Everything is written in C# or ASP and I am developing a desktop app for VB.net. Using the http protocol then I don’t depend on anyone’s library. With that said I will look at Tweetinvi again if my current approach fails. The most important questions I want to ask are simple:

  1. what URL should I be using? I have tried all of these and only gotten a response from '“https://api.twitter.com/oauth” but not sure what it is…

         '"https://api.twitter.com/oauth" <<<<<<<<<<<<<<<<<,,, This works but what is it?
         ' "https://userstream.twitter.com/1.1/user.json"  - could not be resolved
         ' "http://stream.twitter.com/ "- could not be resolved
         '"https://api.twitter.com/oauth/request_token" not found
         '"https://api.twitter.com/oauth/access_token" not found
         '"https://api.twitter.com/1.1/" not found - could not be resolved
         '"https://stream.twitter.com/1.1/" not found
         '"https://api.twitter.com/oauth/authorize" forbidden
         '"https://stream.twitter.com/1.1/" forbidden
         '"https://stream.twitter.com/1.1/statuses/filter.json"  forbidden
    
  2. Would you look at my POST command and maybe that is a way you can tell what I am screwing up… here it is…

Header: "OAuth oauth_nonce=“NjM1NTYwNDQwODExMDUzMDI4”, oauth_signature_method=“HMAC-SHA1”, oauth_timestamp=“1420465281”, oauth_consumer_key=“F1if6ZvPaWcbrhq81leLFNOhT”, oauth_token=“859829466-AdZSBtACvpbQlY1hGgGi0YCVJJG0LgzhEP3HkWoK”, oauth_signature=“Jeq4EzwE3rzrqk5M7m2YZJOCxYM%3D”, oauth_version=“1.0"”

BODY:
"status=Hoping%20to%20be%20authorized"
3. re: Please note that http://localhost:xxxx is not a valid redirection URL for the twitter authentication platform! - I AM USING THIS BECAUSE I DON"T HAVE A WEBSITE…Maybe this is the issue??? What should I change it to??

And maybe the most important part: I am trying to create a streaming API from my own account to my desktop… then programmatically use it. (a widget won’t help)

thanks again Andy

Steve


#4

One of the better walkthroughs I know of is this blog post from my colleague @lfcipriani - the code is in Ruby but well-commented. However, that is not quite your use case as that implements sign-in with Twitter and here you can simply use application authorization to connect to the streaming endpoint.

I modelled this using the twurl utility, and also took a look at the source code for the twitter4j library.

The steps using twurl are basically to do this:

$ twurl authorize --consumer-key=kkkkkk --consumer-secret=aaaaa -t (-t gives us tracing)

POSTs to api.twitter.com/1.1/oauth/request_token which then returns a URL I need to visit to get a PIN. This isn’t quite what you’d do, you can exchange the tokens in code. Either way, you then store away the tokens (which is what ends up in the .twurlrc file for future use by twurl).

Next…

$ twurl -H stream.twitter.com -X POST "/1.1/statuses/filter.json?track=London" -t

POSTs to stream.twitter.com with the filter required on the streaming endpoint (or, you could just GET the sample stream instead). The OAuth header here includes the signature and token from the previous call.

Don’t worry about the redirection URL on the apps.twitter.com site too much.

I can’t see what URL you’re trying to connect to on the streaming call - I may be missing where you’ve defined resource_url and it looks like you are passing in a status= parameter which won’t work with streaming. Those are certainly things I’d look again at.

Hoping this sets you off on the right course! :construction_worker:


#5

Hi Andy,

On a related note, I am trying to access the streaming API, but I cannot properly connect.

This probably has to do with the proxy that I am using - I have created access to https://stream.twitter.com which I presumed was the only one I needed.
However, I still cannot establish connection, so I was wondering which other sites I should create connection to- are the following the only ones I would have to create access to?

https://api.twitter.com
https://upload.twitter.com
https://userstream.twitter.com
https://sitestream.twitter.com


#6

Responded on the other thread.


#7