github-asset fails to upload

[jnweiger]

Affter pip install githubrelease, I am trying to upload a new file into an existing release tag.
I get the following error:

github-asset owncloud/vm upload v8.1.2-16.1-201509032039 ./img/xUbuntu_14.04-owncloud-8.1.2-16.1-201509032039-DEBUG.vmx.zip
release v8.1.2-16.1-201509032039: uploading ./img/xUbuntu_14.04-owncloud-8.1.2-16.1-201509032039-DEBUG.vmx.zip
Traceback (most recent call last):
  File "/usr/bin/github-asset", line 9, in 
    load_entry_point('githubrelease==0.0.0', 'console_scripts', 'github-asset')()
  File "/usr/lib/python2.7/site-packages/github_release.py", line 183, in gh_asset
    return handle_http_error(lambda: commands[args.pop(1)](*args))
  File "/usr/lib/python2.7/site-packages/github_release.py", line 187, in handle_http_error
    func()
  File "/usr/lib/python2.7/site-packages/github_release.py", line 183, in 
    return handle_http_error(lambda: commands[args.pop(1)](*args))
  File "/usr/lib/python2.7/site-packages/github_release.py", line 126, in gh_asset_upload
    data=f.read())
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 335, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 438, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 328, in send
    raise SSLError(e)
requests.exceptions.SSLError: hostname 'uploads.github.com' doesn't match either of '*.githubusercontent.com', 'githubusercontent.com'

[jnweiger]

Workaround:

--- xxx 2015-09-04 01:16:09.704558643 +0200
+++ /usr/lib/python2.7/site-packages/github_release.py  2015-09-04 01:11:46.750523838 +0200
@@ -123,7 +123,7 @@
             response = request('POST', 
                 'https://uploads.github.com/repos/{0}/releases/{1}/assets?name={2}'.format(repo_name, release['id'], basename),
                 headers={'Content-Type':'application/octet-stream'},
-                data=f.read())
+                data=f.read(), verify=False)
             response.raise_for_status()
 
 def gh_asset_erase(repo_name, tag_name, pattern):

[j0057]

Hey Jürgen, I noticed the same thing, but I haven't yet found out what's causing it. It used to work ;-) and comparing Github's API docs to the code, the bug doesn't immediately jump out at me.

[j0057]

verify=False would indicate that it's an SSL certificate problem, right?

[j0057]

Qualys SSL labs gives an A+ rating: https://www.ssllabs.com/ssltest/analyze.html?d=uploads.github.com

[j0057]

OK, I dove into it some more. I created a release, which was overdue:

env/bin/github-release j0057/setuptools-version-command create 2.2
env/bin/github-release j0057/setuptools-version-command debug 2.2
env/bin/github-asset j0057/setuptools-version-command upload 2.2 /var/lib/pypkg27/setuptools-version-command-2.2.tar.gz 
env/bin/github-asset j0057/setuptools-version-command upload 2.2 /var/lib/pypkg27/setuptools_version_command-2.2-py2.py3-none-any.whl 
env/bin/github-release j0057/setuptools-version-command release-notes 2.2
env/bin/github-release j0057/setuptools-version-command publish 2.2
twine upload /var/lib/pypkg27/setuptools-version-command-2.2.tar.gz 
twine upload /var/lib/pypkg27/setuptools_version_command-2.2-py2.py3-none-any.whl

And as you can see on github and on PyPI, it worked.

I guess what I'm saying is: I can't reproduce it, but I agree there is a bug, because I saw it happen too. I hate bugs that magically disappear...

[jnweiger]

Hmm, I am running into more issues, maybe I just don't have the proper setup?
The SSL certificate problem is handled by verify=False, but this leads to warnings.
And then it fails. I admit, it's 600MB, maybe that is too much? The same upload via web-interface succeeds from one machine, but not from another.

uploading ./img/xUbuntu_14.04-owncloud-8.1.2-17.1-201509072043-DEBUG.raw.zip
/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object
 is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.  InsecurePlatformWarning
release v8.1.2-17.1-201509072043: uploading ./img/xUbuntu_14.04-owncloud8.1.2-17.1-201509072043-DEBUG.raw.zip
/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning /usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
Error sending POST to https://uploads.github.com/repos/owncloud/vm/releases/1779018/assets?name=xUbuntu_14.04-owncloud-8.1.2-17.1-201509072043-DEBUG.raw.zip
< POST /repos/owncloud/vm/releases/1779018/assets?name=xUbuntu_14.04-owncloud-8.1.2-17.1-201509072043-DEBUG.raw.zip
< Accept : */*
< Accept-Encoding : gzip, deflate
< Connection : keep-alive
< Content-Length : 660807496
< Content-Type : application/octet-stream
< User-Agent : python-requests/2.7.0 CPython/2.7.6 Linux/3.11.10-29-desktop
< 
< 'PK\x03\x04-\x00\x00\x00\x08\x00Q\xa6\'G\x8cH+"\xff\xff\xff\xff\xff\xff\xff\xff8\x000\x00xUbun' (total 660807496 bytes of data)
  
> 404 Not Found
> Cache-Control : no-cache
> Content-Length : 129            
> 404 Not Found
> Cache-Control : no-cache
> Content-Length : 129
> Content-Security-Policy : default-src 'none'
> Content-Type : application/json; charset=utf-8
> Date : Mon, 07 Sep 2015 21:43:00 GMT
> Strict-Transport-Security : max-age=31557600
> X-Content-Type-Options : nosniff
> X-Frame-Options : deny
> X-Github-Media-Type : github.v3
> X-Github-Request-Id : 6916f9a0-55a9-11e5-8657-a4cbd6e71a9a
> X-Xss-Protection : 1; mode=block
>
> '{"message":"Not Found","request_id"' (total 129 bytes of data)
- done

[j0057]

So in the error you originally posted, you got presented a certificate for githubusercontent.com, but connected to uploads.github.com.

In the second error which you just posted, there are SSL warnings, but ultimately the connection goes through, only the URL posted to appears to be wrong -- or maybe there's a problem with the .netrc?

I tested yesterday against Python 2.7.9 and requests 2.7.0. If I read the documentation of the warning right, upgrading to Python 2.7.9 should fix at least the InsecurePlatformWarnings, is that possible for you? It sounds like it could be related to the original error as well, like if the OpenSSL library you use doesn't support SNI, you could unexpectedly get presented a different certificate.

But I've a feeling that upgrading Python won't make the 404 go away.

Can you maybe upgrade to githubrelease 1.4.3a0 and post the output of github-release owncloud/vm debug $tag -- where $tag is the release version you're creating? That should tell us if it's trying to post to the correct URL or not.

[jcfr]

@jnweiger Let us know if this is still a problem. In the mean time, I am closing this issue.

[Flamefire]

I just ran into the same problem:

Traceback (most recent call last):
File "/home/travis/.local/bin/githubrelease", line 11, in
sys.exit(main())
File "/home/travis/.local/lib/python2.7/site-packages/click/core.py", line 764, in call
return self.main(*args, **kwargs)
File "/home/travis/.local/lib/python2.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/home/travis/.local/lib/python2.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/travis/.local/lib/python2.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/travis/.local/lib/python2.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/travis/.local/lib/python2.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/travis/.local/lib/python2.7/site-packages/click/decorators.py", line 27, in new_func
return f(get_current_context().obj, *args, **kwargs)
File "/home/travis/.local/lib/python2.7/site-packages/github_release.py", line 631, in _cli_asset_upload
gh_asset_upload(*args, **kwargs)
File "/home/travis/.local/lib/python2.7/site-packages/github_release.py", line 116, in with_check_for_credentials
return func(*args, **kwargs)
File "/home/travis/.local/lib/python2.7/site-packages/github_release.py", line 760, in gh_asset_upload
repo_name, tag_name, upload_url, filename, verbose, dry_run)
File "/home/travis/.local/lib/python2.7/site-packages/github_release.py", line 704, in _upload_release_file
data=_ProgressFileReader(f, reporter))
File "/home/travis/.local/lib/python2.7/site-packages/github_release.py", line 65, in _request
response = request(args, **kwargs)
File "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 455, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 558, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 385, in send
raise SSLError(e)
requests.exceptions.SSLError: hostname 'uploads.github.com' doesn't match either of '.githubusercontent.com', 'githubusercontent.com'

Can you reopen this?

[jcfr]

@Flamefire Thanks for reporting the issue. Could you provide more details about the context leading to the error ?

• version of github-release
• version of python
• operating system
• step to reproduce

[Flamefire]

I'm using this on travis: https://travis-ci.com/Flamefire/release-tests/jobs/151523149

• 1.5.8
• 2.7.x (trusty default of travis, trying with 3.6 now)
• trusty
• As per instructions:
  • githubrelease release $TRAVIS_REPO_SLUG create "Release-$TRAVIS_COMMIT" --name "Release for $TRAVIS_COMMIT" --body "BODY cool!" --target-commitish $TRAVIS_COMMIT
  • githubrelease asset $TRAVIS_REPO_SLUG upload "Release-$TRAVIS_COMMIT" $TRAVIS_COMMIT-$TRAVIS_OS_NAME.tar.gz

[Flamefire]

It seems to work for newer python (3.6): https://travis-ci.com/Flamefire/release-tests/jobs/151600878

[jcfr]

2.7.x (trusty default of travis

Thanks for the additional details. I am trying to reproduce now.

[jcfr]

I'm using this on travis: https://travis-ci.com/Flamefire/release-tests/jobs/151523149

  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 558, in send
    r = adapter.send(request, **kwargs)

  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 385, in send
    raise SSLError(e)
requests.exceptions.SSLError: hostname 'uploads.github.com' doesn't match either of '*.githubusercontent.com', 'githubusercontent.com'

I tried to reproduce the problem downloading the docker image associated with the build reporting a failure but I couldn't reproduce the problem.

docker pull travisci/ci-garnet:packer-1512502276-986baf0
docker run -ti --rm travisci/ci-garnet:packer-1512502276-986baf0

And then execute the following without problem:

TRAVIS_REPO_SLUG=jcfr/ci-sandbox
TRAVIS_COMMIT=e73f4efd30e26def13cf1550998bed76dab08599
export GITHUB_TOKEN=XXXXXXXXXXXXX

/opt/python/2.7/bin/pip install githubrelease
githubrelease=/opt/python/2.7/bin/githubrelease

mkdir /work && cd work/
$githubrelease release $TRAVIS_REPO_SLUG create "Release-$TRAVIS_COMMIT" --name "Release for $TRAVIS_COMMIT" --body "BODY cool!" --target-commitish $TRAVIS_COMMIT

$githubrelease release $TRAVIS_REPO_SLUG create "Release-$TRAVIS_COMMIT" --name "Release for $TRAVIS_COMMIT" --body "BODY cool!" --target-commitish $TRAVIS_COMMIT

TRAVIS_OS_NAME=linux
echo "Foo" > $TRAVIS_COMMIT-$TRAVIS_OS_NAME.tar.gz

$githubrelease asset $TRAVIS_REPO_SLUG upload "Release-$TRAVIS_COMMIT" $TRAVIS_COMMIT-$TRAVIS_OS_NAME.tar.gz

[jcfr]

Is this consistently reproducible ? If you are restart the build, does it happen again ?

[Flamefire]

Yes it is. I made a separate test repo for this (https://travis-ci.com/Flamefire/travis_test) and reported this to the travis team (travis-ci/travis-ci#10233)

I'm not very familiar with SSL certificates but I think that there might be some caching breaking this (Github reports wrong certificate, travis stores this and fails to use a newer one) or that it is due to the issues with SSL on Python 2.7:

Collecting githubrelease
/home/travis/.local/lib/python2.7/site-packages/pip/vendor/urllib3/util/ssl.py:369: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
/home/travis/.local/lib/python2.7/site-packages/pip/vendor/urllib3/util/ssl.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
/home/travis/.local/lib/python2.7/site-packages/pip/vendor/urllib3/util/ssl.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning

[jcfr]

Closing. We will be transitioning the continuous integration to GitHub Actions (or CircleCI)