All Systems operational.

Systems

Incidents

2017-04-12 14:15:48 UTC resolved staged-recipes
Conversion blocked

Having issues with Travis CI's API failing repeatedly on us at staged-recipes, which is blocking the conversion of recipes. Please see upstream issue ( https://github.com/travis-ci/travis-ci/issues/6284 ) for more details.

Update 2017-04-17 13:59:40 UTC

Appears this was due to use of an endpoint with Travis CI's API in conda-smithy that requested information about all current feedstocks. We have updated this to use a different endpoint so as to only interact with the feedstock being added in question. Thanks to @isuruf for identifying and proposing the fix to this issue.

xref: https://github.com/conda-forge/conda-smithy/pull/505

2016-08-24 04:46:57 UTC resolved staged-recipes
AppVeyor API blocking conversion

AppVeyor API registration error with itsdangerous is blocking feedstock conversion. Will also result in itsdangerous showing up in PRs. Users are recommended to merge with master and remove itsdangerous from their PRs. Snippet of the traceback shown below. Several restarts of the conversion step have been tried without success.

CI Summary for conda-forge/itsdangerous-feedstock (can take ~30s): * conda-forge/itsdangerous-feedstock already enabled on travis-ci * conda-forge/itsdangerous-feedstock enabled on CircleCI Traceback (most recent call last): File "/home/travis/miniconda/bin/conda-smithy", line 9, in <module> load_entry_point('conda-smithy==1.0.3', 'console_scripts', 'conda-smithy')() File "/home/travis/miniconda/lib/python3.5/site-packages/conda_smithy/cli.py", line 236, in main args.subcommand_func(args) File "/home/travis/miniconda/lib/python3.5/site-packages/conda_smithy/cli.py", line 164, in __call__ ci_register.add_project_to_appveyor(owner, repo) File "/home/travis/miniconda/lib/python3.5/site-packages/conda_smithy/ci_register.py", line 114, in add_project_to_appveyor response.raise_for_status() File "/home/travis/miniconda/lib/python3.5/site-packages/requests/models.py", line 844, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://ci.appveyor.com/api/projects Traceback (most recent call last): File ".CI/create_feedstocks.py", line 186, in <module> subprocess.check_call(['conda', 'smithy', 'register-ci', '--feedstock_directory', feedstock_dir] + owner_info) File "/home/travis/miniconda/lib/python3.5/subprocess.py", line 581, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['conda', 'smithy', 'register-ci', '--feedstock_directory', '/tmp/tmpw0ne44uj__feedstocks/itsdangerous-feedstock', '--organization', 'conda-forge']' returned non-zero exit status 1

cc @pelson

Update 2016-08-24 05:01:50 UTC

After several restarts and a bit of time this resolved itself. However, this is hardly the first time I have seen this issue today or in general with AppVeyor's API. Maybe we need to use a similar strategy to one used with Travis CI's API.

Update 2016-09-03 03:47:25 UTC

Seeing a recurrence of this issue. See this build as an example.

Update 2016-09-03 03:52:28 UTC

In summary: AppVeyor are issuing a http 500 error intermittently when attempting to create a feedstock.

(For the status page, this is probably all that is needed, right @jakirkham?)

Update 2016-09-03 04:39:53 UTC

Yeah I'm still trying to find a healthy balance. In any event staged-recipes is cleared, so closing this out again.

Update 2016-09-30 00:56:59 UTC

xref: https://github.com/appveyor/ci/issues/1045

Update 2017-04-17 14:44:35 UTC

Seeing this again. Maybe appveyor is rate limiting the API

2016-08-24 01:59:49 UTC resolved Travis CI
Travis CI OS X Backlog

Seems that there is some serious issue affecting Travis CI builds and causing them to wait for long periods on the queue. Have raised this issue upstream as ( https://github.com/travis-ci/travis-ci/issues/6515 ) to see if they have any ideas as to what is going on and if we can get it fixed.

For the meantime, I'd like to ask people to nice builds. If you see multiple builds for a PR at staged-recipes, please cancel all older version of it. If you are planning any large running builds, please hold off on them until we can track down this issue. It does seem some builds are getting processed, but it appears to be very slowly and long after they were initiated.

Please bear with us as we try to sort out this issue.

Update 2016-08-24 03:12:44 UTC

Reducing to degraded as we seem to actually have some things running on the queue now. Also looks like the number of jobs running is starting to fall off. Will continue to monitor the situation.

Update 2016-08-24 18:47:04 UTC

Travis CI has now announced that they are stopping all OS X builds due to instability in their OS X infrastructure. Upgrading this to major outage.

Update 2016-08-25 03:12:53 UTC

Adding staged-recipes to this outage as conversions to feedstocks occur on Travis CI. So they will similarly be blocked until Travis CI comes back online.

Update 2016-08-25 18:58:09 UTC

It appears Travis CI is re-enabling OS X builds and are starting to work through the backlog. Downgrading to degraded performance. Will provide more updates as this progresses.

Update 2016-08-26 05:59:34 UTC

Have removed staged-recipes as they have cleared. Feel free to close this when you are comfortable with Travis CI's progress.

Update 2016-08-27 15:16:39 UTC

This seems to be resolved now. Going to go ahead and close this out. If people are still having issues, please let us know.

Update 2017-02-05 18:07:55 UTC

...there seem to be a long backlog on travis of staged-recipes again.

This status issue is on a different problem with Travis CI a long time ago. If there are new issues, I would recommend raising one. Though we are certainly aware of backlog issues with Travis CI that have been occurring recently.

...how do I cancel the expired PRs? There is no such option from my travis login.

Travis CI does not extend that privilege to anyone that does not have write access to the repo. I had put together a script ( https://github.com/conda-forge/conda-forge-build-setup-feedstock/pull/52 ) that would terminate old PR builds early and could be used at staged-recipes ( https://github.com/conda-forge/staged-recipes/pull/2264 ) or feedstocks (once the dust settles on the first two). However, it seems to be stuck in discussion ATM. Feel free to nudge it along if you think it is important.

Update 2017-02-06 04:42:12 UTC

Well this problem seems to show up on a weekly basis during the work week. So IMHO this is a big problem as it mucks with people's ability to get work done. That said, you are right it goes away on the weekend.

There are pros and cons to enabling on your fork. Pro being it is on your own queue vs conda-forge's. Con it still adds to the overall Travis CI queue and it fails to do things like test on the merge commit. Probably not worthwhile to enable Travis CI on your fork in the end.