Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: new golang rest api template #4

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

vdhieu
Copy link

@vdhieu vdhieu commented Jun 21, 2022

No description provided.

@vdhieu vdhieu force-pushed the chore/template-upgrade branch 4 times, most recently from bdc3558 to 10b1b58 Compare June 27, 2022 04:28
@vdhieu vdhieu force-pushed the chore/template-upgrade branch 4 times, most recently from a4d54d3 to e48f9f1 Compare July 4, 2022 05:21
@vdhieu vdhieu force-pushed the chore/template-upgrade branch from e48f9f1 to fd9d874 Compare July 4, 2022 05:23
@vdhieu vdhieu changed the title [WIP] chore: new golang rest api template chore: new golang rest api template Jul 8, 2022
@monotykamary
Copy link

A lot of it looks good. I'm mostly concerned on the best practices for our PostgresStore. This is what we do on some of our projects:

// repo/pg/pg.go
package pg

import (
	"context"
	"fmt"
	"net/http"
	"time"

	"github.com/.../api/config"
	"github.com/.../api/model/errors"
	"github.com/.../api/repo"
	"github.com/.../api/util"
	"go.uber.org/zap"
	"gorm.io/driver/postgres"
	"gorm.io/gorm"
	glog "gorm.io/gorm/logger"
)

// store is implimentation of repository
type store struct {
	database *gorm.DB
}

// DB database connection
func (s *store) DB() *gorm.DB {
	return s.database
}

// NewTransaction for database connection
// return an db instance and a done(error) function, if the error != nil -> rollback
func (s *store) NewTransaction() (newRepo repo.DBRepo, finallyFn repo.FinallyFunc) {
	newDB := s.database.Begin()
	finallyFn = func(err error) error {
		if err != nil {
			nErr := newDB.Rollback().Error
			if nErr != nil {
				return errors.NewStringError(nErr.Error(), http.StatusInternalServerError)
			}
			return errors.NewStringError(err.Error(), util.ParseErrorCode(err))
		}

		cErr := newDB.Commit().Error
		if cErr != nil {
			return errors.NewStringError(cErr.Error(), http.StatusInternalServerError)
		}
		return nil
	}

	return &store{database: newDB}, finallyFn
}

func (s *store) NewTransactionWithContext(ctx context.Context) (newRepo repo.DBRepo, finallyFn repo.FinallyFunc) {
	newDB := s.database.WithContext(ctx).Begin()
	finallyFn = func(err error) error {
		if err != nil {
			nErr := newDB.Rollback().Error
			if nErr != nil {
				return errors.NewStringError(nErr.Error(), http.StatusInternalServerError)
			}
			return errors.NewStringError(err.Error(), util.ParseErrorCode(err))
		}

		cErr := newDB.Commit().Error
		if cErr != nil {
			return errors.NewStringError(cErr.Error(), http.StatusInternalServerError)
		}
		return nil
	}

	return &store{database: newDB}, finallyFn
}

func NewPostgresStore(cfg *config.Config) (repo.DBRepo, func() error) {
	var logLevel glog.LogLevel
	switch cfg.LogLevel {
	case "debug", "DEBUG":
		logLevel = glog.Info
	default:
		logLevel = glog.Warn
	}

	db, err := gorm.Open(postgres.New(postgres.Config{
		DSN: fmt.Sprintf("user=%v password=%v dbname=%v host=%v port=%v sslmode=%v",
			cfg.DBUser, cfg.DBPass, cfg.DBName, cfg.DBHost, cfg.DBPort, cfg.DBSSLMode),
		PreferSimpleProtocol: true, // disables implicit prepared statement usage
	}), &gorm.Config{
		Logger: glog.Default.LogMode(logLevel),
	})
	if err != nil {
		zap.L().Panic("cannot connect to db", zap.Error(err))
	}
	sqlDB, err := db.DB()
	if err != nil {
		zap.L().Panic("cannot connect to db", zap.Error(err))
	}
	/*
		https://github.com/brettwooldridge/HikariCP/wiki/About-Pool-Sizing
		connections = ((core_count * 2) + effective_spindle_count)
		9 connections = ((4 * 2) + 1)
	*/
	sqlDB.SetMaxIdleConns(9)

	/*
		To avoid the server creating new connections outside of the connection
		pool and creating a "too many clients" deadlock situation, we set max
		open connections to less than the default database limit (100). This
		workload is calculated for up to 4 instances: 3 API servers + 1 indexer
		for up to 72 open connections with 28 reserved.
	*/
	sqlDB.SetMaxOpenConns(18)

	/*
		This is set to avoid zombie connections for inactive, but open connections
		by closing them lazily (will not close active connections). These zombie
		connections may occur from an I/O partition or a single upset event.
	*/
	sqlDB.SetConnMaxLifetime(1 * time.Hour)

	return &store{database: db}, sqlDB.Close
}

Our setup contains a few things:

  • A switch case to check our log level to organize verbosity of standard output messages along with the zap logger
  • An additional entry for SSLMODE for databases that terminate TLS at the TCP protocol - cfg.DBSSLMode
  • A method to flatten transactions with context injection (Optional)

With regard to how transactions should be composed (by callback vs applicatively), it should ultimately be up to the developer which style they prefer. The transaction style we use here aims to avoid callback hell for nested transactions (in the event SAVEPOINTs exhibit undesirable behavior. We use it in conjuction with context.WithTimeout as best practice to avoid infinitely long transactions.

	ctx, cancel := context.WithTimeout(context.Background(), consts.DefaultLockTime*time.Second)
	defer cancel()

	tx, done := h.store.NewTransactionWithContext(ctx)

	err = h.repo.Advisory.SetIsolationLevel(tx, "REPEATABLE READ")
	if err != nil {
		zap.L().Sugar().Errorf("h.repo.Advisory.SetIsolationLevel(): %v", err)
		return util.HandleError(c, done(err))
	}

	character, err := h.repo.Character.GetByTokenIdAndOwnerAddressForUpdate(tx, request.TokenId, challengeInfo.Address)
	if err != nil {
		if inerr.Is(err, gorm.ErrRecordNotFound) {
			zap.L().Sugar().Errorf("Character.GetByTokenIdAndOwnerAddressForUpdate() character not found")
			return util.HandleError(c, done(errors.ErrCharacterNotfound))
		}
		zap.L().Sugar().Errorf("Character.GetByTokenIdAndOwnerAddressForUpdate()")
		return util.HandleError(c, done(errors.ErrInternalServerError))
	}

      ...
      done(nil)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants