TwitterAPI: lists/members/create_all with 100 user_ids returns http status 502


#1

What I am doing is:

I create a new list (API: https://dev.twitter.com/docs/api/1/post/lists/create) for a user (my own user), then I retrieve all the people I am following ( ~1500 users with the API: https://dev.twitter.com/docs/api/1/get/friends/ids) and I split in chunks of 100 users to add those users to the new list (API: https://dev.twitter.com/docs/api/1/post/lists/members/create_all) and at the first call I get a 502 HTTP response with the following content:

    <!DOCTYPE html>
    <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
        <meta http-equiv="Content-Language" content="en-us">
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <title>Twitter / Over capacity</title>
        <link href="//si0.twimg.com/favicons/favicon.ico" rel="shortcut icon" type="image/x-icon">
        <link rel="stylesheet" href="//si0.twimg.com/errors/fullscreen_errors.css">
    </head>
    <body>
    <div class="topbar js-topbar">
        <div class="global-nav" data-section-term="top_nav">
            <div class="global-nav-inner">
                <div class="container">
                    <ul class="nav js-global-actions">
                        <li class="home">
                            <a class="nav-logo-link" href="//twitter.com">
                                <i class="bird-topbar-blue"></i>
                            </a>
                        </li>
                    </ul>
                    <div class="pull-right">
                        <ul class="nav secondary-nav">
                            <li><a href="//twitter.com" id="homeLink">Home &rarr;</a></li>
                        </ul>
                    </div>
                </div>
            </div>
        </div>
    </div>
    <div class="blue-sky">
        <div class="body-content">
    
            <div class="canvas">
                <img class= "illustration whale" src= "//si0.twimg.com/errors/whale.png" >
            </div>
            <h1>Twitter is over capacity.</h1>
            <p>Please wait a moment and try again. For more information, check out <a href="http://status.twitter.com">Twitter Status</a>.</p>
    
            <div class="footer">
                <ul class="languages">
    
                    <li><a onclick="displayLanguage('zh-cn');return false;" href="#">简体中文</a></li>
    
                    <li><a onclick="displayLanguage('msa');return false;" href="#">Malay</a></li>
    
                    <li><a onclick="displayLanguage('fil');return false;" href="#">Filipino</a></li>
    
                    <li><a onclick="displayLanguage('fi');return false;" href="#">Suomi</a></li>
    
                    <li><a onclick="displayLanguage('sv');return false;" href="#">Svenska</a></li>
    
                    <li><a onclick="displayLanguage('ja');return false;" href="#">日本語</a></li>
    
                    <li><a onclick="displayLanguage('ko');return false;" href="#">한국어</a></li>
    
                    <li><a onclick="displayLanguage('de');return false;" href="#">Deutsch</a></li>
    
                    <li><a onclick="displayLanguage('it');return false;" href="#">Italiano</a></li>
    
                    <li><a onclick="displayLanguage('pt');return false;" href="#">português (Brasil)</a></li>
    
                    <li><a onclick="displayLanguage('id');return false;" href="#">Indonesian</a></li>
    
                    <li><a onclick="displayLanguage('da');return false;" href="#">Dansk</a></li>
    
                    <li><a onclick="displayLanguage('no');return false;" href="#">Norsk</a></li>
    
                    <li><a onclick="displayLanguage('nl');return false;" href="#">Nederlands</a></li>
    
                    <li><a onclick="displayLanguage('ar');return false;" href="#">العربية</a></li>
    
                    <li><a onclick="displayLanguage('ca');return false;" href="#">català</a></li>
    
                    <li><a onclick="displayLanguage('gl');return false;" href="#">Galego</a></li>
    
                    <li><a onclick="displayLanguage('en');return false;" href="#">English</a></li>
    
                </ul>
                <ul class="links">
                    <li class="first">&copy; 2012 Twitter</li>
                    <li><a href="/about">About</a></li>
                    <li><a href="/help">Help</a></li>
                    <li><a href="http://status.twitter.com/">Status</a></li>
                </ul>
            </div>
        </div>
    </div>
    
    <div class="whale-water"></div>
    
    <!-- BEGIN google analytics -->
    <script type="text/javascript">
    
        var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
        document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
    </script>
    <script type="text/javascript">
        var pageTracker = _gat._getTracker("UA-30775-6");
        pageTracker._setDomainName("twitter.com");
        pageTracker._trackPageview('503 Error');
    
    </script>
    <!-- END google analytics -->
    
    <script type="text/javascript">
        //<![CDATA[
        var twttr = {};
    
        twttr.translations = {
    
            "zh-cn": {
    
                "header": "Twitter 超载了。",
                "description": "请稍候再试。 更多详情,请查看 <a href=\"http://status.twitter.com\">Twitter 状态</a>。",
                "home": "主页 &rarr;"
            },
    
            "msa": {
    
                "header": "Twitter telah melebihi kapasiti.",
                "description": "Sila tunggu sebentar dan cuba lagi. Untuk maklumat lanjut, semak <a href=\"http://status.twitter.com\">Status Twitter</a>.",
                "home": "Utama &rarr;"
            },
    
            "fil": {
    
                "header": "Ang Twitter ay lampas na sa kapasidad.",
                "description": "Mangyaring maghintay ng ilang sandali at subukang muli. Para sa karagdagang impormasyon tingnan ang <a href=\"http://status.twitter.com\">Katayuan ng Twitter</a>.",
                "home": "Home &rarr;"
            },
    
            "fi": {
    
                "header": "",
                "description": "Odota hetki ja yritä uudelleen. Lisätietoja saat <a href=\"http://status.twitter.com\">Twitterin tilasivulta</a>.",
                "home": "Etusivu &rarr;"
            },
    
            "sv": {
    
                "header": "Twitter är överbelastat.",
                "description": "Vänligen vänta en stund och försök igen. För mer information, kolla <a href=\"http://status.twitter.com\">Twitterstatus</a>.",
                "home": "Hem &rarr;"
            },
    
            "ja": {
    
                "header": "Twitterの処理能力の限界を超えました。",
                "description": "しばらくしてからもう一度お試しください。詳しくは<a href=\"http://status.twitter.com\">Twitterステータス</a>をご覧ください。",
                "home": "ホーム &rarr;"
            },
    
            "ko": {
    
                "header": "트위터 용량 초과입니다.",
                "description": "잠시 후 다시 시도 해주세요. 자세한 내용은 <a href=\"http://status.twitter.com\">트위터 시스템 현황</a>에서 확인 가능 합니다.",
                "home": "홈 &rarr;"
            },
    
            "de": {
    
                "header": "Twitter ist überlastet.",
                "description": "Bitte warte einen Moment und versuche es erneut. Für weitere Informationen überprüfe den<a href=\"http://status.twitter.com\">Twitter Status</a>.",
                "home": "Startseite &rarr;"
            },
    
            "it": {
    
                "header": "Twitter è sovraccarico.",
                "description": "Per favore, aspetta un momento e riprova. Per maggiori informazioni visita <a href=\"http://status.twitter.com\">Twitter Status</a>.",
                "home": "Home &rarr;"
            },
    
            "pt": {
    
                "header": "O Twitter está sobrecarregado.",
                "description": "Por favor, aguarde um momento e tente novamente. Para mais informações, confira o <a href=\"http://status.twitter.com\">Status do Twitter</a>.",
                "home": "Início &rarr;"
            },
    
            "id": {
    
                "header": "Twitter sedang kelebihan beban.",
                "description": "Silakan tunggu beberapa saat dan coba lagi. Untuk informasi lebih lanjut, lihat <a href=\"http://status.twitter.com\">Status Twitter</a>.",
                "home": "Beranda &rarr;"
            },
    
            "da": {
    
                "header": "Twitter er overbelastet.",
                "description": "Vent venligst et øjeblik og prøv igen. For mere information, tjek <a href=\"http://status.twitter.com\">Twitter Status</a>.",
                "home": "Hjem &rarr;"
            },
    
            "no": {
    
                "header": "Twitter er overbelastet.",
                "description": "Vennligst prøv igjen om en stund. Sjekk <a href=\"http://status.twitter.com\">Twitters status</a> for mer informasjon.",
                "home": "Hjem &rarr;"
            },
    
            "nl": {
    
                "header": "Twitter is overbelast.",
                "description": "Probeer het later opnieuw. Bezoek <a href=\"http://status.twitter.com\">Twitter-status</a> voor meer informatie.",
                "home": "Startpagina &rarr;"
            },
    
            "ar": {
    
                "header": "تويتر يعاني من الحِمل الزائد.",
                "description": "رجاءً انتظر لحظة وأعِد المحاولة. لمزيد من المعلومات، تفقّد <a href=\"http://status.twitter.com\">حالة تويتر</a>.",
                "home": "الرئيسية &rarr;"
            },
    
            "ca": {
    
                "header": "Twitter està desbordat.",
                "description": "Espera't un moment i torna-ho a provar. Per a més informació, consulta <a href=\"http://status.twitter.com\">Estat de Twitter</a>.",
                "home": "Inici &rarr;"
            },
    
            "gl": {
    
                "header": "O Twitter está sobrecargado.",
                "description": "Por favor agarda un momento e téntao de novo. Se queres máis información, visita <a href=\"http://status.twitter.com\">Estado de Twitter</a>.",
                "home": "Inicio &rarr;"
            },
    
            "en": {
    
                "header": "Twitter is over capacity.",
                "description": "Please wait a moment and try again. For more information, check out <a href=\"http://status.twitter.com\">Twitter Status</a>.",
                "home": "Home &rarr;"
            }
    
        };
    
        function displayLanguage(lang) {
            if (lang && twttr.translations[lang]) {
                document.getElementsByTagName('h1')[0].innerHTML = twttr.translations[lang].header;
                document.getElementsByTagName('p')[0].innerHTML = twttr.translations[lang].description;
                document.getElementById('homeLink').innerHTML = twttr.translations[lang].home;
    
            }
        }
        var lang = window.navigator.language ? window.navigator.language.replace(/^(..).*$/, '$1') : undefined;
        displayLanguage(lang);
        //]]>
    </script>
    </body>
    </html>

I tried this since 5 days ago, many times, and all the time I get this message saying that twitter is over capacity all the times.

With a lower number of ids (like 5-10) sometime it works… but the API call is rate limited.

Is anything wrong with what I am doing?

###Update
Already aware of this bug: issue 383 which looks related (but get 502 instead of 404)


Creating twitter list results in additional list with 0 members
#2

502s happen because it takes us longer to process the request then we’ll wait for being responding. In many cases like these where you’re creating or writing content, you’ll find the action did in fact complete. After you get this error, have you verified whether the list members have been created?

The API write limits are not explicitly rate limited. There’s likely no harm in your dropping from 100 users to something smaller at a time when issuing the write request.


#3

2003809783000140281


#4

When I try to add 100 people at a time to twitter list I get 100% failure rate. The failure rate seems to go down as I lower the number of people I try to add to list at a time.

Seeing as the api says 100 though, maybe twitter could make the limit lower so people don’t run into this error by normal usage.