Any efforts to implement Java client for Gnip 2.0 API

gnip
java

#1

I would like to if anybody working to implement java client for Gnip 2.0 API.
We use Gnip API to provide business solutions to client.
We would like to enhance our services as per 2.0
If anybody already working, we would like to help what ever we can.


#2

Hi there,

I hope you have seen our guidance here: http://support.gnip.com/apis/powertrack2.0/transition.html

Version 1.0 streaming code should work fine with version 2.0. Of course the endpoint URLs have been updated… and if you are using the Backfill feature, there is a slight change there (see the doc above about the details).

The biggest changes are with the Rules API 2.0:

There is a change with the DELETE method when removing rules/filters. See http://support.gnip.com/apis/powertrack2.0/api_reference.html#DeleteRules for the details.

Along with PowerTrack Operator updates, there are some important new features arriving with version 2.0 of the Rules API.

Request payload sizes

When adding rules to PowerTrack, JSON objects are POSTed to the Rules API. The Rules API limits the size of these JSON payloads. Rules API 1.0 has a data request payload size limit of 1 MB. With Rules API 2.0, the data request can be up to 5 MB.

Rule IDs

With version 1.0 a ‘rule’ has two attributes: value and tag. The value attribute contains the syntax of the rule, while the tag is a user-specified string used to either provide a universally-unique ID (UUID) (considered a best practice, btw), or a tag/label to logically group rules (e.g., “rule related to weather projects”).

PowerTrack 2.0 introduces a new rule attribute, a primary key id, which is auto-generated when a rule is created. Unique Rule IDs are important since with PowerTrack 2.0 only rule IDs and tags are included in the gnip.matching_rules metadata. (Note that is similar to version 1 ‘long’ rules behavior with only rule tags included.) PowerTrack 2.0 supports only ‘long’ rules, so the rule syntax (or the rule value) is not returned in the matching rule array. Since the rule syntax is not returned, it is important to have a unique id so the rule syntax can be looked up on the client-side.

Since this new id serves as an UUID, the rule tag no longer needs to play that role. Rule tags remain optional, but can be a convenient mechanism to logically group rules into sets, or add other metadata to a rule object. Example uses of rule tags is to group rules by project, campaign or client.

Here is an example of the new matching rules metadata that is included with all Tweets:

{
  "gnip": {
    "matching_rules": [
      {
        "tag": "Weather monitoring project",
        "id": 714923996685316096
      },
      {
        "tag": "Weather monitoring project",
        "id": 714928183628271616
      },
      {
        "tag": "Smart Cities project",
        "id": 713928144542271723
      }
    ]
  }
}

New Rule Validations

As with previous versions, every rule that is sent to the Rules API has its syntax verified. If you reference a PowerTrack Operator that does not exist, you will receive an error and the request is rejected. Note that if you are uploading an array of 100 rules, a single bad rule will prevent any rules in the request from being added.

With version 2, the following three additional vaidations applied to your rule syntax. These new validations will prevent some of the most common mistakes when writing PowerTrack rules, where a rule is syntactic valid but does not implement the intended logic. The rule examples below illustrate the potential affects of these logical mistakes using recent 30-day Search counts.

No explicit AND logical phrases. This is probably the most common syntax mistake with PowerTrack rules. An unquoted AND (using any case) is treated as a simple keyword and not a logical AND. If any rule has an unquoted AND, it will be rejected.

(climate AND change) winter → 351 Tweets
(climate change) winter → 1,600 Tweets
No lowercase or logical phrases. Logical ORs must be uppercase and unquoted. Unquoted, lowercase or clauses are treated as a simple keyword and not a logical OR. If any rule has an unquoted, lowercase or, it will be rejected.

(snow or cold) weather → 281 Tweets
(snow OR cold) weather → 250,000 Tweets
No NOT logical phrases. Logical NOTs must implemented with a - (dash) character. Unquoted NOT claueses (using any case) are treated as a simple keyword and not a logical NOT. If any rule has an unquoted NOT, it will be rejected.

(sol playa) NOT lang:es → 13 Tweets
(sol playa) -lang:es → 3,300 Tweets

Rule Validation Endpoint

PowerTrack 2.0 provides a rule validation endpoint:

https://gnip-api.twitter.com/rules/powertrack/accounts/<accountName>/<streamLabel>/validation.json

This endpoint enables you to submit candidate rules and check whether the rule has valid syntax or not.


#3

Just a quick update. I recently updated the Twitter Hosebird Client to work with Gnip PowerTrack 2.0. See HERE for the details.