crash when trying to convert image manually

[ptempier]

./acceleration-service/accelctl task create anonymous/hub.docker.com/library/nginx

2024-01-15T14:17:52.105500000Z time="2024-01-15T14:17:52.103805662Z" level=info msg="POST /api/v1/conversions 200 9.432762ms 587>5bytes 172.23.0.1" module=api
2024-01-15T14:17:52.105670000Z time="2024-01-15T14:17:52.104344145Z" level=info msg="pulling image anonymous/hub.docker.com/library/nginx@sha256:35a707bebd92e639a71fec1d5b344a4eb00f113123e38c54a5b84962928f5d08" module=converter
2024-01-15T14:17:52.139351000Z time="2024-01-15T14:17:52.139156808Z" level=info msg="received webhook request from 172.23.0.1:39324" module=api
2024-01-15T14:17:52.168598000Z time="2024-01-15T14:17:52.168251079Z" level=info msg="POST /api/v1/conversions 200 29.081074ms 587>5bytes 172.23.0.1" module=api
2024-01-15T14:17:52.208599000Z time="2024-01-15T14:17:52.208307144Z" level=info msg="received webhook request from 172.23.0.1:39336" module=api
2024-01-15T14:17:52.245770000Z time="2024-01-15T14:17:52.245372522Z" level=info msg="POST /api/v1/conversions 200 37.047493ms 587>5bytes 172.23.0.1" module=api
2024-01-15T14:17:52.276090000Z time="2024-01-15T14:17:52.275572672Z" level=info msg="received webhook request from 172.23.0.1:39340" module=api
2024-01-15T14:17:52.288114000Z time="2024-01-15T14:17:52.287599582Z" level=warning msg="reference for unknown type: application/vnd.in-toto+json" digest="sha256:6a977dec78ebd58b02ad596d5b14c68ace7c3989c944a63ea601082399ff215b" mediatype=application/vnd.in-toto+json size=30079
2024-01-15T14:17:52.288294000Z time="2024-01-15T14:17:52.287710912Z" level=warning msg="reference for unknown type: application/vnd.in-toto+json" digest="sha256:b5c71c53363f4930c0c81a63d5e5e4aa4681e7f3c9b5f24771065f4aef5dbe24" mediatype=application/vnd.in-toto+json size=2552722
2024-01-15T14:17:52.293492000Z time="2024-01-15T14:17:52.292947803Z" level=info msg="POST /api/v1/conversions 200 17.353913ms 587>5bytes 172.23.0.1" module=api
2024-01-15T14:17:52.300763000Z time="2024-01-15T14:17:52.300587106Z" level=info msg="GET /api/v1/conversions 200 1.047785ms 0>3366bytes 127.0.0.1" module=api
2024-01-15T14:17:52.313416000Z time="2024-01-15T14:17:52.313054485Z" level=warning msg="reference for unknown type: application/vnd.in-toto+json" digest="sha256:b4667fb61a2a6ef3fed1bcdfd9dbffa022ed66416bf918fbb01f276e17ff96f5" mediatype=application/vnd.in-toto+json size=30166
2024-01-15T14:17:52.313651000Z time="2024-01-15T14:17:52.313259913Z" level=warning msg="reference for unknown type: application/vnd.in-toto+json" digest="sha256:0aee1be3dc624ed7f544d15327ad2e15b3f261028f95ba6e813182a2f90295bb" mediatype=application/vnd.in-toto+json size=2563209
2024-01-15T14:17:52.438790000Z time="2024-01-15T14:17:52.438478099Z" level=warning msg="reference for unknown type: application/vnd.in-toto+json" digest="sha256:f240738a2112072c68bee74bbebb78259b42bdf846b9d9e367e304f2107adde6" mediatype=application/vnd.in-toto+json size=29954
2024-01-15T14:17:52.552566000Z time="2024-01-15T14:17:52.551780936Z" level=info msg="pulled image anonymous/hub.docker.com/library/nginx@sha256:3bd66d9f2dde2fef40a4e8957a06fb42976137eb337e7496cdc3957220ca2c5a , elapse 505.835195ms" module=converter
2024-01-15T14:17:52.552715000Z time="2024-01-15T14:17:52.551823497Z" level=info msg="converting image anonymous/hub.docker.com/library/nginx@sha256:3bd66d9f2dde2fef40a4e8957a06fb42976137eb337e7496cdc3957220ca2c5a" module=converter
2024-01-15T14:17:52.565620000Z time="2024-01-15T14:17:52.565151773Z" level=info msg="pulled image anonymous/hub.docker.com/library/nginx@sha256:ef7877a6114bccf081ac21439438b699779fbe111940fb1352cabee5b5e92b38 , elapse 483.398754ms" module=converter
2024-01-15T14:17:52.565767000Z time="2024-01-15T14:17:52.565203265Z" level=info msg="converting image anonymous/hub.docker.com/library/nginx@sha256:ef7877a6114bccf081ac21439438b699779fbe111940fb1352cabee5b5e92b38" module=converter
2024-01-15T14:17:52.621251000Z time="2024-01-15T14:17:52.620945645Z" level=info msg="pulled image anonymous/hub.docker.com/library/nginx@sha256:35a707bebd92e639a71fec1d5b344a4eb00f113123e38c54a5b84962928f5d08 , elapse 477.061265ms" module=converter
2024-01-15T14:17:52.621424000Z time="2024-01-15T14:17:52.620993021Z" level=info msg="converting image anonymous/hub.docker.com/library/nginx@sha256:35a707bebd92e639a71fec1d5b344a4eb00f113123e38c54a5b84962928f5d08" module=converter
2024-01-15T14:17:52.638219000Z panic: runtime error: invalid memory address or nil pointer dereference
2024-01-15T14:17:52.638558000Z
2024-01-15T14:17:52.638653000Z runtime/debug.Stack()
2024-01-15T14:17:52.638711000Z /usr/lib/golang/src/runtime/debug/stack.go:24 +0x7a
2024-01-15T14:17:52.638785000Z golang.org/x/sync/singleflight.newPanicError({0x2044c60, 0x2aadcd0})
2024-01-15T14:17:52.638857000Z /root/go/pkg/mod/golang.org/x/[email protected]/singleflight/singleflight.go:44 +0x45
2024-01-15T14:17:52.638943000Z golang.org/x/sync/singleflight.(*Group).doCall.func2.1()
2024-01-15T14:17:52.639099000Z /root/go/pkg/mod/golang.org/x/[email protected]/singleflight/singleflight.go:193 +0x5b
2024-01-15T14:17:52.639187000Z panic({0x2044c60, 0x2aadcd0})
2024-01-15T14:17:52.639252000Z /usr/lib/golang/src/runtime/panic.go:890 +0x262
2024-01-15T14:17:52.639311000Z github.com/goharbor/acceleration-service/pkg/converter.(*Metric).imageSize(0xc00053b3b0, {0x2313be8, 0xc000657d10}, {0x2318aa0, 0xc000100460}, 0x0, {0x2310cb0, 0x2b7aee0})
2024-01-15T14:17:52.639417000Z /root/acceleration-service/pkg/converter/metric.go:61 +0x92
2024-01-15T14:17:52.639515000Z github.com/goharbor/acceleration-service/pkg/converter.(*Metric).SetTargetImageSize(0xc00053b3b0, {0x2313be8, 0xc000657d10}, 0xc000135740, 0x0)
2024-01-15T14:17:52.639570000Z /root/acceleration-service/pkg/converter/metric.go:44 +0xd8
2024-01-15T14:17:52.639777000Z github.com/goharbor/acceleration-service/pkg/converter.(*Converter).Convert(0xc000135740, {0x2313be8, 0xc000657d10}, {0xc002724c80, 0x7f}, {0xc00031b0e0, 0x43}, {0x0, 0x0})
2024-01-15T14:17:52.639884000Z /root/acceleration-service/pkg/converter/converter.go:163 +0x15aa
2024-01-15T14:17:52.639962000Z github.com/goharbor/acceleration-service/pkg/adapter.(*LocalAdapter).Convert(0xc0003ad2f0, {0x2313be8, 0xc000657d10}, {0xc00085e580, 0x7f})
2024-01-15T14:17:52.640045000Z /root/acceleration-service/pkg/adapter/adapter.go:130 +0x69a
2024-01-15T14:17:52.640119000Z github.com/goharbor/acceleration-service/pkg/adapter.(*LocalAdapter).Dispatch.func2.1.1()
2024-01-15T14:17:52.640227000Z /root/acceleration-service/pkg/adapter/adapter.go:164 +0xbc
2024-01-15T14:17:52.640360000Z github.com/goharbor/acceleration-service/pkg/metrics.(*OpWrapper).OpWrap(0xc0001250b0, 0xc00230db88, {0xc000725b50, 0x1, 0x1})
2024-01-15T14:17:52.640423000Z /root/acceleration-service/pkg/metrics/metrics.go:82 +0x9c
2024-01-15T14:17:52.640475000Z github.com/goharbor/acceleration-service/pkg/adapter.(*LocalAdapter).Dispatch.func2.1()
2024-01-15T14:17:52.640517000Z /root/acceleration-service/pkg/adapter/adapter.go:163 +0x133
2024-01-15T14:17:52.640569000Z golang.org/x/sync/singleflight.(*Group).doCall.func2(0xc000725c97, 0xc000101040, 0xc00230df08)
2024-01-15T14:17:52.640616000Z /root/go/pkg/mod/golang.org/x/[email protected]/singleflight/singleflight.go:198 +0xc8
2024-01-15T14:17:52.640690000Z golang.org/x/sync/singleflight.(*Group).doCall(0x2b3e430, 0xc000101040, {0xc00085e580, 0x7f}, 0xc00230df08)
2024-01-15T14:17:52.640849000Z /root/go/pkg/mod/golang.org/x/[email protected]/singleflight/singleflight.go:200 +0x15c
2024-01-15T14:17:52.640900000Z golang.org/x/sync/singleflight.(*Group).Do(0x2b3e430, {0xc00085e580, 0x7f}, 0xc00230df08)
2024-01-15T14:17:52.640971000Z /root/go/pkg/mod/golang.org/x/[email protected]/singleflight/singleflight.go:113 +0x385
2024-01-15T14:17:52.641087000Z github.com/goharbor/acceleration-service/pkg/adapter.(*LocalAdapter).Dispatch.func2()
2024-01-15T14:17:52.641154000Z /root/acceleration-service/pkg/adapter/adapter.go:162 +0xe5
2024-01-15T14:17:52.641207000Z github.com/goharbor/acceleration-service/pkg/adapter.NewWorker.func1()
2024-01-15T14:17:52.641356000Z /root/acceleration-service/pkg/adapter/worker.go:44 +0x5d
2024-01-15T14:17:52.641414000Z created by github.com/goharbor/acceleration-service/pkg/adapter.NewWorker
2024-01-15T14:17:52.641502000Z /root/acceleration-service/pkg/adapter/worker.go:41 +0x13e
2024-01-15T14:17:52.641574000Z
2024-01-15T14:17:52.641618000Z
2024-01-15T14:17:52.641666000Z goroutine 30 [running]:
2024-01-15T14:17:52.641714000Z golang.org/x/sync/singleflight.(*Group).doCall.func1()
2024-01-15T14:17:52.641797000Z /root/go/pkg/mod/golang.org/x/[email protected]/singleflight/singleflight.go:170 +0x4b5
2024-01-15T14:17:52.641861000Z golang.org/x/sync/singleflight.(*Group).doCall(0x2b3e430, 0xc000101040, {0xc00085e580, 0x7f}, 0xc00230df08)
2024-01-15T14:17:52.641910000Z /root/go/pkg/mod/golang.org/x/[email protected]/singleflight/singleflight.go:205 +0x173
2024-01-15T14:17:52.641957000Z golang.org/x/sync/singleflight.(*Group).Do(0x2b3e430, {0xc00085e580, 0x7f}, 0xc00230df08)
2024-01-15T14:17:52.642013000Z /root/go/pkg/mod/golang.org/x/[email protected]/singleflight/singleflight.go:113 +0x385
2024-01-15T14:17:52.642079000Z github.com/goharbor/acceleration-service/pkg/adapter.(*LocalAdapter).Dispatch.func2()
2024-01-15T14:17:52.642225000Z /root/acceleration-service/pkg/adapter/adapter.go:162 +0xe5
2024-01-15T14:17:52.642276000Z github.com/goharbor/acceleration-service/pkg/adapter.NewWorker.func1()
2024-01-15T14:17:52.642321000Z /root/acceleration-service/pkg/adapter/worker.go:44 +0x5d
2024-01-15T14:17:52.642378000Z created by github.com/goharbor/acceleration-service/pkg/adapter.NewWorker
2024-01-15T14:17:52.642425000Z /root/acceleration-service/pkg/adapter/worker.go:41 +0x13e
2024-01-15T14:17:53.917939000Z time="2024-01-15T14:17:53.917486234Z" level=info msg="Version: v0.0.10-36-gfb1bf94 fb1bf94.20240115.1201\n"
2024-01-15T14:17:54.043654000Z time="2024-01-15T14:17:54.043445176Z" level=info msg="[API] HTTP server started on 0.0.0.0:2077"

[Desiki-high]

It looks like nginx@sha256:35a707bebd92e639a71fec1d5b344a4eb00f113123e38c54a5b84962928f5d08 is not a standard OCI V1 image: reference for unknown type: application/vnd.in-toto+json. So get imageSize failed:

acceleration-service/pkg/converter/metric.go

Lines 61 to 86 in b82ccee

	switch image.MediaType {
	case ocispec.MediaTypeImageIndex, images.MediaTypeDockerSchema2ManifestList:
	manifests, err := images.FilterPlatforms(images.ChildrenHandler(cs), platformMC)(ctx, *image)
	if err != nil {
	return imageSize, err
	}
	for _, manifest := range manifests {
	children, err := images.ChildrenHandler(cs)(ctx, manifest)
	if err != nil {
	return imageSize, err
	}
	for _, desc := range children {
	imageSize += desc.Size
	}
	}
	case ocispec.MediaTypeImageManifest, images.MediaTypeDockerSchema2Manifest:
	children, err := images.ChildrenHandler(cs)(ctx, *image)
	if err != nil {
	return imageSize, err
	}
	for _, desc := range children {
	imageSize += desc.Size
	}
	default:
	return imageSize, fmt.Errorf("unknown descriptor type %s", image.MediaType)
	}

[Desiki-high]

I try to pull nginx@sha256:35a707bebd92e639a71fec1d5b344a4eb00f113123e38c54a5b84962928f5d08 failed by nerdctl pull nginx@sha256:35a707bebd92e639a71fec1d5b344a4eb00f113123e38c54a5b84962928f5d08.

WARN[0007] reference for unknown type: application/vnd.in-toto+json 
FATA[0007] failed to unpack image on snapshotter overlayfs: mismatched image rootfs and manifest layers

[imeoer]

@Desiki-high Perhaps we should address the panic as well.