Details
-
Bug
-
Resolution: Unresolved
-
Medium
-
None
-
None
-
None
Description
Frequently, a response "received unexpected HTTP status: 500 Internal Server Error" is received when attempting to push to internal registry, e.g.
docker push registry.kube-system.svc.rec.io:5555/bluval/porter:1.0 fe9a8b4f1dcc: Preparing fe9a8b4f1dcc: Pushed received unexpected HTTP status: 500 Internal Server Error
The image can be found in registry but still the push fails occasionally:
[cloudadmin@controller-1 j]$ docker push registry.kube-system.svc.rec.io:5555/bluval/porter:1.0 The push refers to repository [registry.kube-system.svc.rec.io:5555/bluval/porter] 2475b11e4b59: Layer already exists 44ec04fa5fba: Layer already exists 35b17a8c3a93: Layer already exists 1.0: digest: sha256:824f041b2757f8da2980092ace0991b4b8c6689e76c3b94f677521f7157bc7b8 size: 943 [cloudadmin@controller-1 j]$ docker push registry.kube-system.svc.rec.io:5555/bluval/porter:1.0 The push refers to repository [registry.kube-system.svc.rec.io:5555/bluval/porter] 2475b11e4b59: Layer already exists 44ec04fa5fba: Layer already exists 35b17a8c3a93: Layer already exists 1.0: digest: sha256:824f041b2757f8da2980092ace0991b4b8c6689e76c3b94f677521f7157bc7b8 size: 943 [cloudadmin@controller-1 j]$ docker push registry.kube-system.svc.rec.io:5555/bluval/porter:1.0 The push refers to repository [registry.kube-system.svc.rec.io:5555/bluval/porter] 2475b11e4b59: Layer already exists 44ec04fa5fba: Layer already exists 35b17a8c3a93: Layer already exists received unexpected HTTP status: 500 Internal Server Error [cloudadmin@controller-1 j]$
Log excerpt from a registry pod:
Sep 19 11:31:59 controller-1 dockerd[2869]: time="2019-09-19T11:31:59.427553942+03:00" level=info msg="Attempting next endpoint for push after error: received unexpected HTTP status: 500 Internal Server Error" Sep 19 12:30:22 controller-1 dockerd[2869]: time="2019-09-19T12:30:22.795423817+03:00" level=warning msg="failed to upload schema2 manifest: received unexpected HTTP status: 500 Internal Server Error - falling back to schema1" Sep 19 12:30:22 controller-1 dockerd[2869]: time="2019-09-19T12:30:22.842725367+03:00" level=info msg="Attempting next endpoint for push after error: received unexpected HTTP status: 500 Internal Server Error" Sep 19 12:34:19 controller-1 dockerd[2869]: time="2019-09-19T12:34:19.448281710+03:00" level=warning msg="failed to upload schema2 manifest: received unexpected HTTP status: 500 Internal Server Error - falling back to schema1" Sep 19 12:34:22 controller-1 dockerd[2869]: time="2019-09-19T12:34:22.071897919+03:00" level=info msg="Attempting next endpoint for push after error: received unexpected HTTP status: 500 Internal Server Error" Sep 19 12:36:06 controller-1 dockerd[2869]: time="2019-09-19T12:36:06.332029718+03:00" level=info msg="Attempting next endpoint for push after error: received unexpected HTTP status: 500 Internal Server Error" Sep 19 12:39:53 controller-1 dockerd[2869]: time="2019-09-19T12:39:53.426861752+03:00" level=warning msg="failed to upload schema2 manifest: received unexpected HTTP status: 500 Internal Server Error - falling back to schema1" Sep 19 12:39:53 controller-1 dockerd[2869]: time="2019-09-19T12:39:53.867425401+03:00" level=info msg="Attempting next endpoint for push after error: received unexpected HTTP status: 500 Internal Server Error" Sep 19 12:41:21 controller-1 dockerd[2869]: time="2019-09-19T12:41:21.114190515+03:00" level=warning msg="failed to upload schema2 manifest: received unexpected HTTP status: 500 Internal Server Error - falling back to schema1" Sep 19 12:41:21 controller-1 dockerd[2869]: time="2019-09-19T12:41:21.180209597+03:00" level=info msg="Attempting next endpoint for push after error: received unexpected HTTP status: 500 Internal Server Error"
Product release:
[cloudadmin@controller-1 j]$ cat /etc/product-release release=TA-1 build=126
