Newbie how to partition data best practices? multiple streaming connections

filter
streaming

#1

Hi

I have been working on a POC that use stream status/filter api (https://dev.twitter.com/streaming/reference/post/statuses/filter)

I need to collect data for several different applications. Ideally I would like start a separate ‘collector’ for each application. This would make it easy for the collector to know why it received data (ie it matched a specific filter), and what do with the data. In our case we need to process data ‘follows’ differently than ‘tags’. The data for different apps must be stored in separate secure silos.

based on my limited understanding it seems like I can only have a single collector running. It appears that filter predicates are logically or’ed together, making it difficult to know why any given tweet was received. I.E. I do not think I can insure the filters are mutually exclusive

How do people typically solve this problem?

kind regards