Pushing image of more than 30GB fails. Need help.

[prashi2202]

I have configured a harbor registry on EKS cluster with RDS and S3 Bucket at the backend. We are trying to push a large image (30 - 100GB) into the repository by running docker push and the same fails with below error message on registry replicaset.

The overall resource quota for the vas namespace is set to 8 GB and each pod is set to use resources as mentioned below.

Need a expert help to understand ..

Why the image push is failing, I have tried the same on normal filesystem (non s3) setup the image got pushed without issues?
Is there anything that am missing on Harbor values file or anything that need to be set at S3 Bucket side to avoid issues while pushing large images?
What is the decent size of memory that need to be configured on the registry pod/on the whole of harbor to ensure the pods dont fail during parallel upload of large/small images ?

Error:

Name: harbor-registry-7988676864.169ed0c8b716b7b9
Message: Error creating: pods "harbor-registry-7988676864-tjzdr" is forbidden: exceeded quota: resource-quota-xxx, requested: limits.memory=2Gi,requests.memory=2Gi, used: limits.memory=7296Mi,requests.memory=7296Mi, limited: limits.memory=8Gi,requests.memory=8Gi

Logs:

NAME READY STATUS RESTARTS AGE
harbor-chartmuseum-7c8f9cbb55-46mn8 1/1 Running 0 43h == 128Mi and 100m CPU limit with replica set to 1
harbor-core-5d4f6ff85-mnzxv 1/1 Running 0 42h == 128Mi and 100m CPU limit with replica set to 1
harbor-jobservice-68d965b75b-b9sm7 1/1 Running 0 41h == 128Mi and 100m CPU limit with replica set to 2
harbor-jobservice-68d965b75b-ssjxb 1/1 Running 1 43h
harbor-notary-server-f5798db99-rsnht 1/1 Running 0 42h == 128Mi and 100m CPU limit with replica set to 1
harbor-notary-signer-6d4b6768f-9g778 1/1 Running 0 43h == 128Mi and 100m CPU limit with replica set to 1
harbor-portal-5647cf8d45-jv4jw 1/1 Running 0 22h == 128Mi and 100m CPU limit with replica set to 1
harbor-redis-0 1/1 Running 0 43h == 128Mi and 100m CPU limit with replica set to 1
harbor-registry-66c88cb8f9-hnsq7 2/2 Running 0 43h == 2GB memory and 100m CPU limit with replica set to 2.
harbor-registry-676b988d75-h2hnm 2/2 Running 0 16h
harbor-trivy-0 1/1 Running 0 43h == 128Mi and 100m CPU limit with replica set to 1

In the values.yaml file I have already set the following

ingress annotations :

annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "0"
ingress.kubernetes.io/proxy-body-size: "0"

S3 Bucket config:

s3:
region: us-west-2
bucket:
chunksize: 52428800
multipartcopychunksize: 33554432
multipartcopymaxconcurrency: 100
multipartcopythresholdsize: 33554432

Resource config:

For all pods except registry:

resources:
requests:
memory: 128Mi
cpu: 50m
limits:
memory: 128Mi
cpu: 100m

Registry container :

resources:
requests:
memory: 1024Mi
cpu: 100m
limits:
memory: 1024Mi
cpu: 100m

Registry controller container:

resources:
requests:
memory: 1024Mi
cpu: 100m
limits:
memory: 1024Mi
cpu: 100m

[prashi2202]

I was able to resolve this issue by adding the below annotation to the ingress part.

nginx.ingress.kubernetes.io/proxy-request-buffering: "off"