HTTP status code 500 appears randomly


We have been struggling getting the cards to show up all the time.
For example this post (as a gallery):

When I try to validate it randomly it shows the HTTP status code 500 appearing when I press refresh a few times suddenly one out of a few tries shows everything green and all is good. When I post on twitter it shows no Twitter Card.

Are we doing something wrong?