Skip to content

Commit

Permalink
v5.3.4 (#644)
Browse files Browse the repository at this point in the history
* fix: improve model ID field customization (#604)

Updates places where `"id"` was hardcoded instead of using `model.IDField()`.

* Ensure uninitialized map is initialized when unmarshaling json
Add tests for this scenario

* exclude migration_table_name from connection string

* add test for OptionsString

* Add support for pointer FKs when preloading a belongs_to association (#602)

* feat: support context-aware tablenames (#614)

This patch adds a feature which enables pop to pass down the connection context to the model's TableName() function by implementing TableName(ctx context.Context) string. The context can be used to dynamically generate tablenames which can be important for prefixed or generic tables and other use cases.

* Bump pg deps (#616)

* Reset to development

* bumping pgx and pgconn versions

Co-authored-by: Stanislas Michalak <[email protected]>
Co-authored-by: Larry M Jordan <[email protected]>

* Latest from master (#620)

* Latest from development (#617)

* fix: improve model ID field customization (#604)

Updates places where `"id"` was hardcoded instead of using `model.IDField()`.

* Ensure uninitialized map is initialized when unmarshaling json
Add tests for this scenario

* exclude migration_table_name from connection string

* add test for OptionsString

* Add support for pointer FKs when preloading a belongs_to association (#602)

* feat: support context-aware tablenames (#614)

This patch adds a feature which enables pop to pass down the connection context to the model's TableName() function by implementing TableName(ctx context.Context) string. The context can be used to dynamically generate tablenames which can be important for prefixed or generic tables and other use cases.

* Bump pg deps (#616)

* Reset to development

* bumping pgx and pgconn versions

Co-authored-by: Stanislas Michalak <[email protected]>
Co-authored-by: Larry M Jordan <[email protected]>

Co-authored-by: Patrik <[email protected]>
Co-authored-by: Michael Montgomery <[email protected]>
Co-authored-by: kyrozetera <[email protected]>
Co-authored-by: Reggie Riser <[email protected]>
Co-authored-by: hackerman <[email protected]>
Co-authored-by: Stanislas Michalak <[email protected]>
Co-authored-by: Larry M Jordan <[email protected]>

* adding goreleaser syntaz (#619)

Co-authored-by: Patrik <[email protected]>
Co-authored-by: Michael Montgomery <[email protected]>
Co-authored-by: kyrozetera <[email protected]>
Co-authored-by: Reggie Riser <[email protected]>
Co-authored-by: hackerman <[email protected]>
Co-authored-by: Stanislas Michalak <[email protected]>
Co-authored-by: Larry M Jordan <[email protected]>

* Resolve issues in UPDATE and DELETE when using schemas (#618)

* Resolve MySQL issues and improve test migrations
* Bump CockroachDB to maintained and supported versions
Version 2.1 has reached EoL in 2019

Signed-off-by: aeneasr <[email protected]>

* Use `PaginatorPageKey` and `PaginatorPerPageKey` variables (#615)

* update pagination_test

* Pass Time structure into timestamp update functions. (#625)

Closes #624

* Allow nullable JSONB and resolve MySQL regression (#639)

* Allow passing args to `Order` (#630)

* Added connection maximum idle time configuration (#635)

This PR add the possibility to configure the connection maximum idle time (https://golang.org/pkg/database/sql/#DB.SetConnMaxIdleTime).

Closes #632

BREAKING CHANGE: Requires Go 1.15 from now on.

* Bump sqlite to 3.35.4 / 1.14.7 (#642)

* Update pg, pgx, sqlx (#643)

- `jackc/pgx` to  version `v4.11.0`.
- `jmoiron/sqlx` to version`v1.3.3`
- `lib/pq` to version`v1.10.1`

* Fix Inner has many associations when passing on multiple arguments (#633)

* Fix Inner has many associations when passing on multiple arguments for inner fields

* Fix broken tests

* clean up extractFieldAndInnerFields function

Co-authored-by: hackerman <[email protected]>
Co-authored-by: Antonio Pagano <[email protected]>

* Remove many to many TX condition for EagerPreload (#645)

* Remove the need to use Tx when loading many to many associations

* replace TX access to create a new tx.Store.Transaction() object

* Added fix/tests for has_many with pointer foreign key (#647)

Co-authored-by: Antonio Pagano <[email protected]>

Co-authored-by: Patrik <[email protected]>
Co-authored-by: Michael Montgomery <[email protected]>
Co-authored-by: kyrozetera <[email protected]>
Co-authored-by: Reggie Riser <[email protected]>
Co-authored-by: hackerman <[email protected]>
Co-authored-by: Stanislas Michalak <[email protected]>
Co-authored-by: Larry M Jordan <[email protected]>
Co-authored-by: Brian Buchholz <[email protected]>
Co-authored-by: Mike Pontillo <[email protected]>
Co-authored-by: Benjamin Blattberg <[email protected]>
Co-authored-by: Jonathan Duck <[email protected]>
Co-authored-by: Arthur Knoepflin <[email protected]>
  • Loading branch information
13 people authored May 6, 2021
1 parent 5a8f51c commit 33f2d5c
Show file tree
Hide file tree
Showing 32 changed files with 592 additions and 175 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
name: Release
runs-on: ubuntu-latest
container:
image: bepsays/ci-goreleaser:1.13-4
image: bepsays/ci-goreleaser:1.15.1
steps:
- name: Checkout Code
uses: actions/checkout@master
Expand All @@ -24,4 +24,4 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GORELEASER_GITHUB_TOKEN }}
with:
version: latest
args: release --rm-dist
args: release --rm-dist
22 changes: 11 additions & 11 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v1
with:
go-version: 1.13
go-version: 1.15
id: go
- name: Checkout Code
uses: actions/checkout@v1
Expand Down Expand Up @@ -64,7 +64,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v1
with:
go-version: 1.13
go-version: 1.15
id: go
- name: Checkout Code
uses: actions/checkout@v1
Expand Down Expand Up @@ -97,7 +97,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v1
with:
go-version: 1.13
go-version: 1.15
id: go
- name: Checkout Code
uses: actions/checkout@v1
Expand All @@ -110,12 +110,12 @@ jobs:
run: |
mkdir -p crdb/certs
pushd crdb
wget -qO- https://binaries.cockroachdb.com/cockroach-v2.1.0.linux-amd64.tgz | tar zxv
mv cockroach-v2.1.0.linux-amd64/cockroach .
wget -qO- https://binaries.cockroachdb.com/cockroach-v20.2.4.linux-amd64.tgz | tar zxv
mv cockroach-v20.2.4.linux-amd64/cockroach .
./cockroach cert create-ca --certs-dir certs --ca-key key
./cockroach cert create-client root --certs-dir certs --ca-key key
./cockroach cert create-node localhost 127.0.0.1 `hostname -s` `hostname -f` --certs-dir certs --ca-key key
./cockroach start --certs-dir certs --listen-addr localhost --port 26259 --http-port 8089 --background
./cockroach start-single-node --certs-dir certs --listen-addr localhost --port 26259 --http-port 8089 --background
popd
- name: Build and run soda
env:
Expand All @@ -139,7 +139,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v1
with:
go-version: 1.13
go-version: 1.15
id: go
- name: Checkout Code
uses: actions/checkout@v1
Expand All @@ -152,9 +152,9 @@ jobs:
run: |
mkdir -p crdb
pushd crdb
wget -qO- https://binaries.cockroachdb.com/cockroach-v2.1.0.linux-amd64.tgz | tar zxv
mv cockroach-v2.1.0.linux-amd64/cockroach .
./cockroach start --insecure --background
wget -qO- https://binaries.cockroachdb.com/cockroach-v20.2.4.linux-amd64.tgz | tar zxv
mv cockroach-v20.2.4.linux-amd64/cockroach .
./cockroach start-single-node --insecure --background
popd
- name: Build and run soda
env:
Expand All @@ -181,7 +181,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v1
with:
go-version: 1.13
go-version: 1.15
id: go
- name: Checkout Code
uses: actions/checkout@v1
Expand Down
2 changes: 1 addition & 1 deletion associations/association.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ func (a *associationComposite) InnerAssociations() InnerAssociations {
// association for Song.
type InnerAssociation struct {
Name string
Fields string
Fields []string
}

// InnerAssociations is a group of InnerAssociation.
Expand Down
39 changes: 29 additions & 10 deletions associations/associations_for_struct.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,29 +34,39 @@ func ForStruct(s interface{}, fields ...string) (Associations, error) {
}
fields = trimFields(fields)
associations := Associations{}
innerAssociations := InnerAssociations{}
fieldsWithInnerAssociation := map[string]InnerAssociations{}

// validate if fields contains a non existing field in struct.
// and verify is it has inner associations.
for i := range fields {
var innerField, field string
var innerField string

if !validAssociationExpRegexp.MatchString(fields[i]) {
return associations, fmt.Errorf("association '%s' does not match the format %s", fields[i], "'<field>' or '<field>.<nested-field>'")
}

if strings.Contains(fields[i], ".") {
dotIndex := strings.Index(fields[i], ".")
field = fields[i][:dotIndex]
innerField = fields[i][dotIndex+1:]
fields[i] = field
}
fields[i], innerField = extractFieldAndInnerFields(fields[i])

if _, ok := t.FieldByName(fields[i]); !ok {
return associations, fmt.Errorf("field %s does not exist in model %s", fields[i], t.Name())
}

if innerField != "" {
innerAssociations = append(innerAssociations, InnerAssociation{fields[i], innerField})
var found bool
innerF, _ := extractFieldAndInnerFields(innerField)

for j := range fieldsWithInnerAssociation[fields[i]] {
f, _ := extractFieldAndInnerFields(fieldsWithInnerAssociation[fields[i]][j].Fields[0])
if innerF == f {
fieldsWithInnerAssociation[fields[i]][j].Fields = append(fieldsWithInnerAssociation[fields[i]][j].Fields, innerField)
found = true
break
}
}

if !found {
fieldsWithInnerAssociation[fields[i]] = append(fieldsWithInnerAssociation[fields[i]], InnerAssociation{fields[i], []string{innerField}})
}
}
}

Expand All @@ -79,7 +89,7 @@ func ForStruct(s interface{}, fields ...string) (Associations, error) {
modelType: t,
modelValue: v,
popTags: tags,
innerAssociations: innerAssociations,
innerAssociations: fieldsWithInnerAssociation[f.Name],
}

a, err := builder(params)
Expand Down Expand Up @@ -121,3 +131,12 @@ func fieldIgnoredIn(fields []string, field string) bool {
}
return true
}

func extractFieldAndInnerFields(field string) (string, string) {
if !strings.Contains(field, ".") {
return field, ""
}

dotIndex := strings.Index(field, ".")
return field[:dotIndex], field[dotIndex+1:]
}
6 changes: 5 additions & 1 deletion connection.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,10 @@ import (
"sync/atomic"
"time"

"github.com/pkg/errors"

"github.com/gobuffalo/pop/v5/internal/defaults"
"github.com/gobuffalo/pop/v5/internal/randx"
"github.com/pkg/errors"
)

// Connections contains all available connections
Expand Down Expand Up @@ -117,6 +118,9 @@ func (c *Connection) Open() error {
if details.ConnMaxLifetime > 0 {
db.SetConnMaxLifetime(details.ConnMaxLifetime)
}
if details.ConnMaxIdleTime > 0 {
db.SetConnMaxIdleTime(details.ConnMaxIdleTime)
}
if details.Unsafe {
db = db.Unsafe()
}
Expand Down
2 changes: 2 additions & 0 deletions connection_details.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ type ConnectionDetails struct {
IdlePool int
// Defaults to 0 "unlimited". See https://golang.org/pkg/database/sql/#DB.SetConnMaxLifetime
ConnMaxLifetime time.Duration
// Defaults to 0 "unlimited". See https://golang.org/pkg/database/sql/#DB.SetConnMaxIdleTime
ConnMaxIdleTime time.Duration
// Defaults to `false`. See https://godoc.org/github.com/jmoiron/sqlx#DB.Unsafe
Unsafe bool
Options map[string]string
Expand Down
2 changes: 1 addition & 1 deletion dialect_cockroach.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ func (p *cockroach) Update(s store, model *Model, cols columns.Columns) error {
}

func (p *cockroach) Destroy(s store, model *Model) error {
stmt := p.TranslateSQL(fmt.Sprintf("DELETE FROM %s WHERE %s", p.Quote(model.TableName()), model.whereID()))
stmt := p.TranslateSQL(fmt.Sprintf("DELETE FROM %s AS %s WHERE %s", p.Quote(model.TableName()), model.alias(), model.whereID()))
_, err := genericExec(s, stmt, model.ID())
return err
}
Expand Down
4 changes: 2 additions & 2 deletions dialect_common.go
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ func genericCreate(s store, model *Model, cols columns.Columns, quoter quotable)
}

func genericUpdate(s store, model *Model, cols columns.Columns, quoter quotable) error {
stmt := fmt.Sprintf("UPDATE %s SET %s WHERE %s", quoter.Quote(model.TableName()), cols.Writeable().QuotedUpdateString(quoter), model.whereNamedID())
stmt := fmt.Sprintf("UPDATE %s AS %s SET %s WHERE %s", quoter.Quote(model.TableName()), model.alias(), cols.Writeable().QuotedUpdateString(quoter), model.whereNamedID())
log(logging.SQL, stmt, model.ID())
_, err := s.NamedExec(stmt, model.Value)
if err != nil {
Expand All @@ -109,7 +109,7 @@ func genericUpdate(s store, model *Model, cols columns.Columns, quoter quotable)
}

func genericDestroy(s store, model *Model, quoter quotable) error {
stmt := fmt.Sprintf("DELETE FROM %s WHERE %s", quoter.Quote(model.TableName()), model.whereID())
stmt := fmt.Sprintf("DELETE FROM %s AS %s WHERE %s", quoter.Quote(model.TableName()), model.alias(), model.whereID())
_, err := genericExec(s, stmt, model.ID())
if err != nil {
return err
Expand Down
9 changes: 6 additions & 3 deletions dialect_mysql.go
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,9 @@ func (m *mysql) Update(s store, model *Model, cols columns.Columns) error {
}

func (m *mysql) Destroy(s store, model *Model) error {
return errors.Wrap(genericDestroy(s, model, m), "mysql destroy")
stmt := fmt.Sprintf("DELETE FROM %s WHERE %s = ?", m.Quote(model.TableName()), model.IDField())
_, err := genericExec(s, stmt, model.ID())
return errors.Wrap(err, "mysql destroy")
}

func (m *mysql) SelectOne(s store, model *Model, query Query) error {
Expand Down Expand Up @@ -155,9 +157,10 @@ func (m *mysql) FizzTranslator() fizz.Translator {

func (m *mysql) DumpSchema(w io.Writer) error {
deets := m.Details()
cmd := exec.Command("mysqldump", "-d", "-h", deets.Host, "-P", deets.Port, "-u", deets.User, fmt.Sprintf("--password=%s", deets.Password), deets.Database)
// Github CI is currently using mysql:5.7 but the mysqldump version doesn't seem to match
cmd := exec.Command("mysqldump", "--column-statistics=0", "-d", "-h", deets.Host, "-P", deets.Port, "-u", deets.User, fmt.Sprintf("--password=%s", deets.Password), deets.Database)
if deets.Port == "socket" {
cmd = exec.Command("mysqldump", "-d", "-S", deets.Host, "-u", deets.User, fmt.Sprintf("--password=%s", deets.Password), deets.Database)
cmd = exec.Command("mysqldump", "--column-statistics=0", "-d", "-S", deets.Host, "-u", deets.User, fmt.Sprintf("--password=%s", deets.Password), deets.Database)
}
return genericDumpSchema(deets, cmd, w)
}
Expand Down
2 changes: 1 addition & 1 deletion dialect_postgresql.go
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ func (p *postgresql) Update(s store, model *Model, cols columns.Columns) error {
}

func (p *postgresql) Destroy(s store, model *Model) error {
stmt := p.TranslateSQL(fmt.Sprintf("DELETE FROM %s WHERE %s", p.Quote(model.TableName()), model.whereID()))
stmt := p.TranslateSQL(fmt.Sprintf("DELETE FROM %s AS %s WHERE %s", p.Quote(model.TableName()), model.alias(), model.whereID()))
_, err := genericExec(s, stmt, model.ID())
if err != nil {
return err
Expand Down
5 changes: 2 additions & 3 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,9 @@ services:
volumes:
- ./sqldumps:/docker-entrypoint-initdb.d
cockroach:
image: cockroachdb/cockroach:v2.1.0
image: cockroachdb/cockroach:v20.2.4
ports:
- "26257:26257"
- "8080:8080"
volumes:
- "./cockroach-data/roach1:/cockroach/cockroach-data"
command: start --insecure
command: start-single-node --insecure
12 changes: 8 additions & 4 deletions executors.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package pop

import (
"reflect"
"time"

"github.com/gobuffalo/pop/v5/associations"
"github.com/gobuffalo/pop/v5/columns"
Expand Down Expand Up @@ -234,8 +235,9 @@ func (c *Connection) Create(model interface{}, excludeColumns ...string) error {
cols.Remove(excludeColumns...)
}

m.touchCreatedAt()
m.touchUpdatedAt()
now := nowFunc().Truncate(time.Microsecond)
m.setUpdatedAt(now)
m.setCreatedAt(now)

if err = c.Dialect.Create(c.Store, m, cols); err != nil {
return err
Expand Down Expand Up @@ -357,7 +359,8 @@ func (c *Connection) Update(model interface{}, excludeColumns ...string) error {
cols.Remove(excludeColumns...)
}

m.touchUpdatedAt()
now := nowFunc().Truncate(time.Microsecond)
m.setUpdatedAt(now)

if err = c.Dialect.Update(c.Store, m, cols); err != nil {
return err
Expand Down Expand Up @@ -401,7 +404,8 @@ func (c *Connection) UpdateColumns(model interface{}, columnNames ...string) err
}
cols.Remove("id", "created_at")

m.touchUpdatedAt()
now := nowFunc().Truncate(time.Microsecond)
m.setUpdatedAt(now)

if err = c.Dialect.Update(c.Store, m, cols); err != nil {
return err
Expand Down
6 changes: 5 additions & 1 deletion finders.go
Original file line number Diff line number Diff line change
Expand Up @@ -273,12 +273,16 @@ func (q *Query) eagerDefaultAssociations(model interface{}) error {
return err
}

if err == sql.ErrNoRows {
continue
}

// load all inner associations.
innerAssociations := association.InnerAssociations()
for _, inner := range innerAssociations {
v = reflect.Indirect(reflect.ValueOf(model)).FieldByName(inner.Name)
innerQuery := Q(query.Connection)
innerQuery.eagerFields = []string{inner.Fields}
innerQuery.eagerFields = inner.Fields
err = innerQuery.eagerAssociations(v.Addr().Interface())
if err != nil {
return err
Expand Down
40 changes: 40 additions & 0 deletions finders_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -908,3 +908,43 @@ func Test_FindManyToMany(t *testing.T) {
r.NoError(err)
})
}

func Test_FindMultipleInnerHasMany(t *testing.T) {
if PDB == nil {
t.Skip("skipping integration tests")
}
transaction(func(tx *Connection) {
r := require.New(t)

user := User{Name: nulls.NewString("Mark")}
err := tx.Create(&user)
r.NoError(err)

book := Book{Title: "Pop Book", Isbn: "PB1", UserID: nulls.NewInt(user.ID)}
err = tx.Create(&book)
r.NoError(err)

writer := Writer{Name: "Jhon", BookID: book.ID}
err = tx.Create(&writer)
r.NoError(err)

friend := Friend{FirstName: "Frank", LastName: "Kafka", WriterID: writer.ID}
err = tx.Create(&friend)
r.NoError(err)

address := Address{Street: "St 27", HouseNumber: 27, WriterID: writer.ID}
err = tx.Create(&address)
r.NoError(err)

u := User{}
err = tx.Eager("Books.Writers.Addresses", "Books.Writers.Friends").Find(&u, user.ID)
r.NoError(err)

r.Len(u.Books, 1)
r.Len(u.Books[0].Writers, 1)
r.Len(u.Books[0].Writers[0].Addresses, 1)
r.Equal(u.Books[0].Writers[0].Addresses[0].HouseNumber, 27)
r.Len(u.Books[0].Writers[0].Friends, 1)
r.Equal(u.Books[0].Writers[0].Friends[0].FirstName, "Frank")
})
}
Loading

0 comments on commit 33f2d5c

Please sign in to comment.