The Account Activity API is for events related to an account i.e. direct messages, likes, etc. It would not help you in this scenario (it is positioned to replace user and site streams, not the public stream with filter).
There’s no operator to remove retweets or replies on the filter endpoint. The enterprise streaming API (PowerTrack) does include the ability to filter out, or to positively include, retweets and replies. I cannot say if that feature will extend to the standard public stream in the future.