If I am following few users through streaming api-public streams. Do I get all the tweets by those users starting from the day i begin following those users through streaming API?
You get the tweets they issue during the time you’re connected. Close the connection and you stop getting the tweets. If they don’t tweet (or related tweets aren’t streamed) while you’re connected, you might get nothing.