The object model in the data dictionary in the twitter documentation does not have field lengths. Why is this? What values should I use when creating a traditional RDBMS schema?
I can work this out by trial and error, of course, but it seems like a lot of unnecessary work, particularly given the large number of optional fields and expansions that are available.
1 Like
This is useful feedback, and we could look into this for a future update to the documentation.
The OpenAPI specification does have string and integer bounds and sizes (in the schema sections) for the various fields, which you may find useful. However, the specification is updated as new endpoints are added, so we generally suggest that you avoid hard-coding expectations / or code with the expectation that additional fields may be added in the future.
2 Likes
@Kim_Prince If you end up with a good generic schema for twitter data, it would be great to share it! We were looking into it for twarc but never completed it sqlite schema · Issue #379 · DocNow/twarc · GitHub
1 Like
