Skip to content

Commit

Permalink
Switch from compress/gzip to klauspost/pgzip
Browse files Browse the repository at this point in the history
Container layers are generally quite large (>1MB). The README
of `klauspost/pgzip` provides benchmarks that display significant
improvements in decompression time over the stdlib (~2x).
  • Loading branch information
kylecarbs committed Nov 2, 2023
1 parent dbcd01c commit 81a4e67
Show file tree
Hide file tree
Showing 35 changed files with 9,144 additions and 6 deletions.
3 changes: 2 additions & 1 deletion cmd/crane/cmd/flatten.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,12 @@
package cmd

import (
"compress/gzip"
"encoding/json"
"fmt"
"log"

gzip "github.com/klauspost/pgzip"

"github.com/google/go-containerregistry/pkg/crane"
"github.com/google/go-containerregistry/pkg/logs"
"github.com/google/go-containerregistry/pkg/name"
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ require (
github.com/docker/docker v24.0.0+incompatible
github.com/google/go-cmp v0.5.9
github.com/klauspost/compress v1.16.5
github.com/klauspost/pgzip v1.2.6
github.com/mitchellh/go-homedir v1.1.0
github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.1.0-rc3
Expand Down
2 changes: 2 additions & 0 deletions go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion internal/gzip/zip.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,10 @@ package gzip
import (
"bufio"
"bytes"
"compress/gzip"
"io"

gzip "github.com/klauspost/pgzip"

"github.com/google/go-containerregistry/internal/and"
)

Expand Down
3 changes: 2 additions & 1 deletion pkg/v1/stream/layer.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ package stream

import (
"bufio"
"compress/gzip"
"crypto"
"encoding/hex"
"errors"
Expand All @@ -26,6 +25,8 @@ import (
"os"
"sync"

gzip "github.com/klauspost/pgzip"

v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/types"
)
Expand Down
3 changes: 2 additions & 1 deletion pkg/v1/tarball/layer.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,13 @@ package tarball

import (
"bytes"
"compress/gzip"
"fmt"
"io"
"os"
"sync"

gzip "github.com/klauspost/pgzip"

"github.com/containerd/stargz-snapshotter/estargz"
"github.com/google/go-containerregistry/internal/and"
comp "github.com/google/go-containerregistry/internal/compression"
Expand Down
3 changes: 2 additions & 1 deletion pkg/v1/tarball/layer_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,12 @@ package tarball

import (
"bytes"
"compress/gzip"
"io"
"os"
"testing"

gzip "github.com/klauspost/pgzip"

"github.com/containerd/stargz-snapshotter/estargz"
"github.com/google/go-containerregistry/internal/compare"
"github.com/google/go-containerregistry/pkg/compression"
Expand Down
3 changes: 2 additions & 1 deletion pkg/v1/validate/layer.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,15 @@ package validate

import (
"archive/tar"
"compress/gzip"
"crypto"
"encoding/hex"
"errors"
"fmt"
"io"
"strings"

gzip "github.com/klauspost/pgzip"

v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
)
Expand Down
Loading

0 comments on commit 81a4e67

Please sign in to comment.