README.md

lochness

Package lochness provides primitives for orchestrating Mistify Agents.

Loch Ness is a simple virtual machine manager ("cloud controller"). It is a proof-of-concept, straight forward implementation for testing various ideas. It is targeted at clusters of up to around 100 physical machines.

Data Model

A Hypervisor is a physical machine

A subnet is an actual IP subnet with a range of usable IP addresses. A hypervisor can have one of more subnets, while a subnet can span multiple hypervisors. This assumes a rather simple network layout.

A Network is a logical collection of subnets. It is generally used by guest creation to decide what logical segment to place a guest on.

A flavor is a virtual resource "Template" for guest creation. A guest has a single flavor.

A FW Group is a collection of firewall rules for incoming IP traffic. A Guest has a single fwgroup.

A guest is a virtual machine. At creation time, a network, fwgroup, and network is required.

Usage

const AgentPort int = 8080

AgentPort is the default port on which to attempt contacting an agent

var (
	// ConfigPath is the path in the config store.
	ConfigPath = "lochness/config/"
)

var DefaultCandidateFunctions = []CandidateFunction{
	CandidateIsAlive,
	CandidateHasSubnet,
	CandidateHasResources,
	CandidateRandomize,
}

DefaultCandidateFunctions is a default list of CandidateFunctions for general use

var (
	// FWGroupPath is the path in the config store
	FWGroupPath = "lochness/fwgroups/"
)

var (
	// FlavorPath is the path in the config store
	FlavorPath = "lochness/flavors/"
)

var (
	// GuestPath is the path in the config store
	GuestPath = "lochness/guests/"
)

var (
	// HypervisorPath is the path in the config store
	HypervisorPath = "lochness/hypervisors/"
)

var (
	// NetworkPath is the path in the config store.
	NetworkPath = "lochness/networks/"
)

var (
	// SubnetPath is the key prefix for subnets
	SubnetPath = "lochness/subnets/"
)

var (
	// VLANGroupPath is the path in the config store for VLAN groups
	VLANGroupPath = "lochness/vlangroups/"
)

var (
	// VLANPath is the path in the config store for VLANs
	VLANPath = "lochness/vlans/"
)

func GetHypervisorID

func GetHypervisorID() string

GetHypervisorID gets the hypervisor id as set with SetHypervisorID. It does not make an attempt to discover the id if not set.

func SetHypervisorID

func SetHypervisorID(id string) (string, error)

SetHypervisorID sets the id of the current hypervisor. It should be used by all daemons that are ran on a hypervisor and are expected to interact with the data stores directly. Passing in a blank string will fall back to first checking the environment variable "HYPERVISOR_ID" and then using the hostname. ID must be a valid UUID. ID will be lowercased.

type Agent

type Agent interface {
	GetGuest(string) (*client.Guest, error)
	CreateGuest(string) (string, error)
	DeleteGuest(string) (string, error)
	GuestAction(string, string) (string, error)
	CheckJobStatus(string, string) (bool, error)
}

Agent is an interface that allows for communication with a hypervisor agent

type CandidateFunction

type CandidateFunction func(*Guest, Hypervisors) (Hypervisors, error)

CandidateFunction is used to select hypervisors that can run the given guest.

type Context

type Context struct {
}

Context carries around data/structs needed for operations

func NewContext

func NewContext(kv kv.KV) *Context

NewContext creates a new context

func (*Context) FWGroup

func (c *Context) FWGroup(id string) (*FWGroup, error)

FWGroup fetches a FWGroup from the config store

func (*Context) FirstHypervisor

func (c *Context) FirstHypervisor(f func(*Hypervisor) bool) (*Hypervisor, error)

FirstHypervisor will return the first hypervisor for which the function returns true.

func (*Context) Flavor

func (c *Context) Flavor(id string) (*Flavor, error)

Flavor fetches a single Flavor from the config store

func (*Context) ForEachConfig

func (c *Context) ForEachConfig(f func(key, val string) error) error

ForEachConfig will run f on each config. It will stop iteration if f returns an error.

func (*Context) ForEachGuest

func (c *Context) ForEachGuest(f func(*Guest) error) error

ForEachGuest will run f on each Guest. It will stop iteration if f returns an error.

func (*Context) ForEachHypervisor

func (c *Context) ForEachHypervisor(f func(*Hypervisor) error) error

ForEachHypervisor will run f on each Hypervisor. It will stop iteration if f returns an error.

func (*Context) ForEachSubnet

func (c *Context) ForEachSubnet(f func(*Subnet) error) error

ForEachSubnet will run f on each Subnet. It will stop iteration if f returns an error.

func (*Context) ForEachVLAN

func (c *Context) ForEachVLAN(f func(*VLAN) error) error

ForEachVLAN will run f on each VLAN. It will stop iteration if f returns an error.

func (*Context) ForEachVLANGroup

func (c *Context) ForEachVLANGroup(f func(*VLANGroup) error) error

ForEachVLANGroup will run f on each VLAN. It will stop iteration if f returns an error.

func (*Context) GetConfig

func (c *Context) GetConfig(key string) (string, error)

GetConfig gets a single value from the config store. The key can contain slashes ("/")

func (*Context) Guest

func (c *Context) Guest(id string) (*Guest, error)

Guest fetches a Guest from the config store

func (*Context) Hypervisor

func (c *Context) Hypervisor(id string) (*Hypervisor, error)

Hypervisor fetches a Hypervisor from the config store.

func (*Context) IsKeyNotFound

func (c *Context) IsKeyNotFound(err error) bool

IsKeyNotFound is a helper to determine if the error is a key not found error

func (*Context) Network

func (c *Context) Network(id string) (*Network, error)

Network fetches a Network from the data store.

func (*Context) NewFWGroup

func (c *Context) NewFWGroup() *FWGroup

NewFWGroup creates a new, blank FWGroup

func (*Context) NewFlavor

func (c *Context) NewFlavor() *Flavor

NewFlavor creates a blank Flavor

func (*Context) NewGuest

func (c *Context) NewGuest() *Guest

NewGuest create a new blank Guest

func (*Context) NewHypervisor

func (c *Context) NewHypervisor() *Hypervisor

NewHypervisor create a new blank Hypervisor.

func (*Context) NewMistifyAgent

func (context *Context) NewMistifyAgent(port int) *MistifyAgent

NewMistifyAgent creates a new MistifyAgent instance within the context

func (*Context) NewNetwork

func (c *Context) NewNetwork() *Network

NewNetwork creates a new, blank Network.

func (*Context) NewSubnet

func (c *Context) NewSubnet() *Subnet

NewSubnet creates a new "blank" subnet. Fill in the needed values and then call Save.

func (*Context) NewVLAN

func (c *Context) NewVLAN() *VLAN

NewVLAN creates a new blank VLAN.

func (*Context) NewVLANGroup

func (c *Context) NewVLANGroup() *VLANGroup

NewVLANGroup creates a new blank VLANGroup.

func (*Context) SetConfig

func (c *Context) SetConfig(key, val string) error

SetConfig sets a single value from the config store. The key can contain slashes ("/")

func (*Context) Subnet

func (c *Context) Subnet(id string) (*Subnet, error)

Subnet fetches a single subnet by ID

func (*Context) VLAN

func (c *Context) VLAN(tag int) (*VLAN, error)

VLAN fetches a VLAN from the data store.

func (*Context) VLANGroup

func (c *Context) VLANGroup(id string) (*VLANGroup, error)

VLANGroup fetches a VLAN from the data store.

type ErrorHTTPCode

type ErrorHTTPCode struct {
	Expected int
	Code     int
}

ErrorHTTPCode should be used for errors resulting from an http response code not matching the expected code

func (ErrorHTTPCode) Error

func (e ErrorHTTPCode) Error() string

Error returns a string error message

type FWGroup

type FWGroup struct {
	ID       string            `json:"id"`
	Metadata map[string]string `json:"metadata"`
	Rules    FWRules           `json:"rules"`
}

FWGroup represents a group of firewall rules

func (FWGroup) MarshalJSON

func (f FWGroup) MarshalJSON() ([]byte, error)

MarshalJSON is a helper for marshalling a FWGroup

func (*FWGroup) Refresh

func (f *FWGroup) Refresh() error

Refresh reloads from the data store

func (*FWGroup) Save

func (f *FWGroup) Save() error

Save persists a FWGroup. It will call Validate.

func (*FWGroup) UnmarshalJSON

func (f *FWGroup) UnmarshalJSON(input []byte) error

UnmarshalJSON is a helper for unmarshalling a FWGroup

func (*FWGroup) Validate

func (f *FWGroup) Validate() error

Validate ensures a FWGroup has reasonable data.

type FWGroups

type FWGroups []*FWGroup

FWGroups is an alias to FWGroup slices

type FWRule

type FWRule struct {
	Source    *net.IPNet `json:"source,omitempty"`
	Group     string     `json:"group"`
	PortStart uint       `json:"portStart"`
	PortEnd   uint       `json:"portEnd"`
	Protocol  string     `json:"protocol"`
	Action    string     `json:"action"`
}

FWRule represents a single firewall rule

type FWRules

type FWRules []*FWRule

FWRules is an alias to a slice of *FWRule

type Flavor

type Flavor struct {
	ID       string            `json:"id"`
	Image    string            `json:"image"`
	Metadata map[string]string `json:"metadata"`
	Resources
}

Flavor defines the virtual resources for a guest

func (*Flavor) Refresh

func (f *Flavor) Refresh() error

Refresh reloads from the data store

func (*Flavor) Save

func (f *Flavor) Save() error

Save persists a Flavor. It will call Validate.

func (*Flavor) Validate

func (f *Flavor) Validate() error

Validate ensures a Flavor has reasonable data. It currently does nothing.

type Flavors

type Flavors []*Flavor

Flavors is an alias to a slice of *Flavor

type Guest

type Guest struct {
	ID           string            `json:"id"`
	Metadata     map[string]string `json:"metadata"`
	Type         string            `json:"type"`       // type of guest. currently just kvm
	FlavorID     string            `json:"flavor"`     // resource flavor
	HypervisorID string            `json:"hypervisor"` // hypervisor. may be blank if not assigned yet
	NetworkID    string            `json:"network"`
	SubnetID     string            `json:"subnet"`
	FWGroupID    string            `json:"fwgroup"`
	VLANGroupID  string            `json:"vlangroup"`
	MAC          net.HardwareAddr  `json:"mac"`
	IP           net.IP            `json:"ip"`
	Bridge       string            `json:"bridge"`
}

Guest is a virtual machine

func (*Guest) Candidates

func (g *Guest) Candidates(f ...CandidateFunction) (Hypervisors, error)

Candidates returns a list of Hypervisors that may run this Guest.

func (*Guest) Destroy

func (g *Guest) Destroy() error

Destroy removes a guest

func (*Guest) MarshalJSON

func (g *Guest) MarshalJSON() ([]byte, error)

MarshalJSON is a helper for marshalling a Guest

func (*Guest) Refresh

func (g *Guest) Refresh() error

Refresh reloads from the data store

func (*Guest) Save

func (g *Guest) Save() error

Save persists the Guest to the data store.

func (*Guest) UnmarshalJSON

func (g *Guest) UnmarshalJSON(input []byte) error

UnmarshalJSON is a helper for unmarshalling a Guest

func (*Guest) Validate

func (g *Guest) Validate() error

Validate ensures a Guest has reasonable data.

type Guests

type Guests []*Guest

Guests is an alias to a slice of *Guest

type Hypervisor

type Hypervisor struct {
	ID                 string            `json:"id"`
	Metadata           map[string]string `json:"metadata"`
	IP                 net.IP            `json:"ip"`
	Netmask            net.IP            `json:"netmask"`
	Gateway            net.IP            `json:"gateway"`
	MAC                net.HardwareAddr  `json:"mac"`
	TotalResources     Resources         `json:"total_resources"`
	AvailableResources Resources         `json:"available_resources"`

	// Config is a set of key/values for driving various config options. writes should
	// only be done using SetConfig
	Config map[string]string
}

Hypervisor is a physical box on which guests run

func (*Hypervisor) AddGuest

func (h *Hypervisor) AddGuest(g *Guest) error

AddGuest adds a Guest to the Hypervisor. It reserves an IPaddress for the Guest. It also updates the Guest.

func (*Hypervisor) AddSubnet

func (h *Hypervisor) AddSubnet(s *Subnet, bridge string) error

AddSubnet adds a subnet to a Hypervisor.

func (*Hypervisor) Destroy

func (h *Hypervisor) Destroy() error

Destroy removes a hypervisor. The Hypervisor must not have any guests.

func (*Hypervisor) ForEachGuest

func (h *Hypervisor) ForEachGuest(f func(*Guest) error) error

ForEachGuest will run f on each Guest. It will stop iteration if f returns an error.

func (*Hypervisor) Guests

func (h *Hypervisor) Guests() []string

Guests returns a slice of GuestIDs assigned to the Hypervisor.

func (*Hypervisor) Heartbeat

func (h *Hypervisor) Heartbeat(ttl time.Duration) error

Heartbeat announces the availability of a hypervisor. In general, this is useful for service announcement/discovery. Should be ran from the hypervisor, or something monitoring it.

func (*Hypervisor) IsAlive

func (h *Hypervisor) IsAlive() bool

IsAlive returns true if the heartbeat is present.

func (*Hypervisor) MarshalJSON

func (h *Hypervisor) MarshalJSON() ([]byte, error)

MarshalJSON is a helper for marshalling a Hypervisor

func (*Hypervisor) Refresh

func (h *Hypervisor) Refresh() error

Refresh reloads a Hypervisor from the data store.

func (*Hypervisor) RemoveGuest

func (h *Hypervisor) RemoveGuest(g *Guest) error

RemoveGuest removes a guest from the hypervisor. Also releases the IP.

func (*Hypervisor) RemoveSubnet

func (h *Hypervisor) RemoveSubnet(s *Subnet) error

RemoveSubnet removes a subnet from a Hypervisor.

func (*Hypervisor) Save

func (h *Hypervisor) Save() error

Save persists a FWGroup. It will call Validate.

func (*Hypervisor) SetConfig

func (h *Hypervisor) SetConfig(key, value string) error

SetConfig sets a single Hypervisor Config value. Set value to "" to unset.

func (*Hypervisor) Subnets

func (h *Hypervisor) Subnets() map[string]string

Subnets returns the subnet/bridge mappings for a Hypervisor.

func (*Hypervisor) UnmarshalJSON

func (h *Hypervisor) UnmarshalJSON(input []byte) error

UnmarshalJSON is a helper for unmarshalling a Hypervisor

func (*Hypervisor) UpdateResources

func (h *Hypervisor) UpdateResources() error

UpdateResources syncs Hypervisor resource usage to the data store. It should only be ran on the actual hypervisor.

func (*Hypervisor) Validate

func (h *Hypervisor) Validate() error

Validate ensures a Hypervisor has reasonable data. It currently does nothing.

func (*Hypervisor) VerifyOnHV

func (h *Hypervisor) VerifyOnHV() error

VerifyOnHV verifies that it is being ran on hypervisor with same hostname as id.

type Hypervisors

type Hypervisors []*Hypervisor

Hypervisors is an alias to a slice of *Hypervisor

func CandidateHasResources

func CandidateHasResources(g *Guest, hs Hypervisors) (Hypervisors, error)

CandidateHasResources returns Hypervisors that have available resources based on the request Flavor of the Guest.

func CandidateHasSubnet

func CandidateHasSubnet(g *Guest, hs Hypervisors) (Hypervisors, error)

CandidateHasSubnet returns Hypervisors that have subnets with available addresses in the request Network of the Guest.

func CandidateIsAlive

func CandidateIsAlive(g *Guest, hs Hypervisors) (Hypervisors, error)

CandidateIsAlive returns Hypervisors that are "alive" based on heartbeat

func CandidateRandomize

func CandidateRandomize(g *Guest, hs Hypervisors) (Hypervisors, error)

CandidateRandomize shuffles the list of Hypervisors.

type MistifyAgent

type MistifyAgent struct {
}

MistifyAgent is an Agent that communicates with a hypervisor agent to perform actions relating to guests

func (*MistifyAgent) CheckJobStatus

func (agent *MistifyAgent) CheckJobStatus(guestID, jobID string) (bool, error)

CheckJobStatus looks up whether a guest job has been completed or not.

func (*MistifyAgent) CreateGuest

func (agent *MistifyAgent) CreateGuest(guestID string) (string, error)

CreateGuest tries to create a new guest on a hypervisor selected from a list of viable candidates

func (*MistifyAgent) DeleteGuest

func (agent *MistifyAgent) DeleteGuest(guestID string) (string, error)

DeleteGuest deletes a guest from a hypervisor

func (*MistifyAgent) FetchImage

func (agent *MistifyAgent) FetchImage(guestID string) (string, error)

FetchImage fetches a disk image that can be used for guest creation

func (*MistifyAgent) GetGuest

func (agent *MistifyAgent) GetGuest(guestID string) (*client.Guest, error)

GetGuest retrieves information on a guest from an agent

func (*MistifyAgent) GuestAction

func (agent *MistifyAgent) GuestAction(guestID, actionName string) (string, error)

GuestAction is used to run various actions on a guest under a hypervisor Actions: "shutdown", "reboot", "restart", "poweroff", "start", "suspend"

type Network

type Network struct {
	ID       string            `json:"id"`
	Metadata map[string]string `json:"metadata"`
}

Network is a logical collection of subnets.

func (*Network) AddSubnet

func (n *Network) AddSubnet(s *Subnet) error

AddSubnet adds a Subnet to the Network.

func (*Network) Refresh

func (n *Network) Refresh() error

Refresh reloads the Network from the data store.

func (*Network) RemoveSubnet

func (n *Network) RemoveSubnet(s *Subnet) error

RemoveSubnet removes a subnet from the network

func (*Network) Save

func (n *Network) Save() error

Save persists a Network. It will call Validate.

func (*Network) Subnets

func (n *Network) Subnets() []string

Subnets returns the IDs of the Subnets associated with the network.

func (*Network) Validate

func (n *Network) Validate() error

Validate ensures a Network has reasonable data. It currently does nothing.

type Networks

type Networks []*Network

Networks is an alias to a slice of *Network

type Resources

type Resources struct {
	Memory uint64 `json:"memory"` // memory in MB
	Disk   uint64 `json:"disk"`   // disk in MB
	CPU    uint32 `json:"cpu"`    // virtual cpus
}

Resources represents compute resources

type Subnet

type Subnet struct {
	ID         string            `json:"id"`
	Metadata   map[string]string `json:"metadata"`
	NetworkID  string            `json:"network"`
	Gateway    net.IP            `json:"gateway"`
	CIDR       *net.IPNet        `json:"cidr"`
	StartRange net.IP            `json:"start"` // first usable IP in range
	EndRange   net.IP            `json:"end"`   // last usable IP in range
}

Subnet is an actual ip subnet for assigning addresses

func (*Subnet) Addresses

func (s *Subnet) Addresses() map[string]string

Addresses returns used IP addresses.

func (*Subnet) AvailableAddresses

func (s *Subnet) AvailableAddresses() []net.IP

AvailableAddresses returns the available ip addresses. this is probably a horrible idea for ipv6.

func (*Subnet) Delete

func (s *Subnet) Delete() error

Delete removes a subnet. It does not ensure it is unused, so use with extreme caution.

func (*Subnet) MarshalJSON

func (s *Subnet) MarshalJSON() ([]byte, error)

MarshalJSON is used by the json package

func (*Subnet) Refresh

func (s *Subnet) Refresh() error

Refresh reloads the Subnet from the data store.

func (*Subnet) ReleaseAddress

func (s *Subnet) ReleaseAddress(ip net.IP) error

ReleaseAddress releases an address. This does not change any thing that may also be referring to this address.

func (*Subnet) ReserveAddress

func (s *Subnet) ReserveAddress(id string) (net.IP, error)

ReserveAddress reserves an ip address. The id is a guest id.

func (*Subnet) Save

func (s *Subnet) Save() error

Save persists the subnet to the datastore.

func (*Subnet) UnmarshalJSON

func (s *Subnet) UnmarshalJSON(input []byte) error

UnmarshalJSON is used by the json package

func (*Subnet) Validate

func (s *Subnet) Validate() error

Validate ensures the values are reasonable.

type Subnets

type Subnets []*Subnet

Subnets is an alias to a slice of *Subnet

type VLAN

type VLAN struct {
	Tag         int    `json:"tag"`
	Description string `json:"description"`
}

VLAN devines the virtual lan for a guest interface

func (*VLAN) Destroy

func (v *VLAN) Destroy() error

Destroy removes the VLAN

func (*VLAN) Refresh

func (v *VLAN) Refresh() error

Refresh reloads the VLAN from the data store.

func (*VLAN) Save

func (v *VLAN) Save() error

Save persists a VLAN. It will call Validate.

func (*VLAN) VLANGroups

func (v *VLAN) VLANGroups() []string

VLANGroups returns the IDs of the VLANGroups associated with the VLAN

func (*VLAN) Validate

func (v *VLAN) Validate() error

Validate ensures a VLAN has resonable data.

type VLANGroup

type VLANGroup struct {
	ID          string            `json:"id"`
	Description string            `json:"description"`
	Metadata    map[string]string `json:"metadata"`
}

VLANGroup defines a set of VLANs for a guest interface

func (*VLANGroup) AddVLAN

func (vg *VLANGroup) AddVLAN(vlan *VLAN) error

AddVLAN adds a VLAN to the VLANGroup

func (*VLANGroup) Destroy

func (vg *VLANGroup) Destroy() error

Destroy removes a VLANGroup

func (*VLANGroup) Refresh

func (vg *VLANGroup) Refresh() error

Refresh reloads the VLAN from the data store.

func (*VLANGroup) RemoveVLAN

func (vg *VLANGroup) RemoveVLAN(vlan *VLAN) error

RemoveVLAN removes a VLAN from the VLANGroup

func (*VLANGroup) Save

func (vg *VLANGroup) Save() error

Save persists a VLANgroup. It will call Validate.

func (*VLANGroup) VLANs

func (vg *VLANGroup) VLANs() []int

VLANs returns the Tags of the VLANs associated with the VLANGroup

func (*VLANGroup) Validate

func (vg *VLANGroup) Validate() error

Validate ensures a VLANGroup has resonable data.

type VLANGroups

type VLANGroups []*VLANGroup

VLANGroups is an alias to a slice of *VLANGroup

type VLANs

type VLANs []*VLAN

VLANs is an alias to a slice of *VLAN

-- Generated with godocdown