1
0
Fork 0
mirror of https://github.com/portainer/portainer.git synced 2025-07-24 07:49:41 +02:00

feat(helm/templates): helm app templates EE-943 (#5449)

* feat(helm): add helm chart backport to ce EE-1409 (#5425)

* EE-1311 Helm Chart Backport from EE

* backport to ce

Co-authored-by: Matt Hook <hookenz@gmail.com>

* feat(helm): list and configure helm chart (#5431)

* backport and tidyup code

* --amend

* using rocket icon for charts

* helm chart bugfix - clear category button

* added matomo analytics for helm chart install

* fix web editor exit warning without changes

* editor modified exit bugfix

* fixed notifications typo

* updated helm template text

* helper text to convey slow helm templates load

Co-authored-by: zees-dev <dev.786zshan@gmail.com>

* removing redundant time-consuming api call by using prop attribute

* feat(helm) helm chart backport from ee EE-1311 (#5436)

* Add missing defaultHelmRepoUrl and mock testing

* Backport EE-1477

* Backport updates to helm tests from EE

* add https by default changes and ssl to tls renaming from EE

* Port install integration test. Disabled by default to pass CI checks

* merged changes from EE for the integration test

* kube proxy whitelist updated to support internal helm install command

Co-authored-by: zees-dev <dev.786zshan@gmail.com>

* Pull in all changes from tech review in EE-943

* added helm to sidebar after rebase, sync CE with EE

* bugfix: kubectl shell not opening - bearer token bug

* tidy go modules & remove yarn-error.log

* removed redundant handler (not used) - to match EE

* resolved merge conflicts, updated code

* feat(helm/views): helm release and application views EE-1236  (#5529)

* feat(helm): add helm chart backport to ce EE-1409 (#5425)

* EE-1311 Helm Chart Backport from EE

* backport to ce

Co-authored-by: Matt Hook <hookenz@gmail.com>

* Pull in all changes from tech review in EE-943

* added helm to sidebar after rebase, sync CE with EE

* removed redundant handler (not used) - to match EE

* feat(helm) display helm charts - backend EE-1236

* copy over components for new applications view EE-1236

* Add new applications datatable component

* Add more migrated files

* removed test not applicable to CE

* baclkported EE app data table code to CE

* removed redundant helm repo url

* resolved conflicts, updated code

* using endpoint middleware

* PR review fixes

* using constants, openapi updated

Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: zees-dev <dev.786zshan@gmail.com>

* fixed test conflicts, go linted

* feat(helm/templates-add): helm templates add repo for user support EE-1278 (#5514)

* feat(helm): add helm chart backport to ce EE-1409 (#5425)

* EE-1311 Helm Chart Backport from EE

* backport to ce

Co-authored-by: Matt Hook <hookenz@gmail.com>

* feat(helm) helm chart backport from ee EE-1311 (#5436)

* Add missing defaultHelmRepoUrl and mock testing

* Backport EE-1477

* Backport updates to helm tests from EE

* add https by default changes and ssl to tls renaming from EE

* Port install integration test. Disabled by default to pass CI checks

* merged changes from EE for the integration test

* kube proxy whitelist updated to support internal helm install command

Co-authored-by: zees-dev <dev.786zshan@gmail.com>

* Pull in all changes from tech review in EE-943

* feat(helm): add helm chart backport to ce EE-1409 (#5425)

* EE-1311 Helm Chart Backport from EE

* backport to ce

Co-authored-by: Matt Hook <hookenz@gmail.com>

* Pull in all changes from tech review in EE-943

* added helm to sidebar after rebase, sync CE with EE

* backport EE-1278, squashed, diffed, updated

* helm install openapi spec update

* resolved conflicts, updated code

* - matching ee codebase at 0afe57034449ee0e9f333d92c252a13995a93019
- helm install using endpoint middleware
- remove trailing slash from added/persisted helm repo urls

* feat(helm) use libhelm url validator and improved path assembly EE-1554 (#5561)

* feat(helm/userrepos) fix getting global repo for ordinary users EE-1562 (#5567)

* feat(helm/userrepos) fix getting global repo for ordinary users EE-1562

* post review changes and further backported changes from EE

* resolved conflicts, updated code

* fixed helm_install handler unit test

* user cannot add existing repo if suffix is '/' (#5571)

* feat(helm/docs) fix broken swagger docs EE-1278 (#5572)

* Fix swagger docs

* minor correction

* fix(helm): migrating code from user handler to helm handler (#5573)

* - migrated user_helm_repos to helm endpoint handler
- migrated api operations from user factory/service to helm factory/service
- passing endpointId into helm service/factory as endpoint provider is deprecated

* upgrade libhelm to hide secrets

Co-authored-by: Matt Hook <hookenz@gmail.com>

* removed duplicate file - due to merge conflict

* dependency injection in helm factory

Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: Matt Hook <hookenz@gmail.com>

* kubernetes.templates -> kubernetes.templates.helm name conflict fix

* Validate the URL added as a public helm repo (#5579)

* fix(helm): helm app deletion fix EE-1581 (#5582)

* updated helm lib to show correct error on uninstall failure

* passing down helm app namespace on deletion

* fix(k8s): EE-1591 non-admin users cannot deploy charts containing secrets (#5590)

Co-authored-by: Simon Meng <simon.meng@portainer.io>

* fix(helm): helm epic bugfixes EE-1582 EE-1593 (#5585)

* - trim trailing slash and lowercase before persisting helm repo
- browser helm templates url /kubernetes/templates/templates -> /kubernetes/templates/helm
- fix publish url
- fix helm repo add refresh
- semi-fix k8s app expansion

* Tidy up swagger documentation related to helm. Make json consistent

* fixed helm release page for non-default namespaces

* k8s app view table expansion bugfix

* EE-1593: publish url load balancer fallback

Co-authored-by: Matt Hook <hookenz@gmail.com>

* k8s app list fix for charts with deployments containing multiple pods - which use the same label (#5599)

* fix(kubernetes): app list view fix for secrets with long keys or values EE-1600 (#5600)

* k8s app secrets key value text overflow ellipses

* wrapping key value pairs instead of ellipses

* fix(helm): helm apps bundling issue across different namespaces EE-1619 (#5602)

* helm apps bundling issue across different namespaces

* - code comments and indentation to ease reading
- moved namespace calc out of loop

* feat(helm/test) disable slow helm search test by default EE-1599 (#5598)

* skip helm_repo_search as it's an integration test

* switch to portainer built in integration test checker

* make module order match EE

* don't print test struct out when skipping integration test

Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: Matt Hook <hookenz@gmail.com>
Co-authored-by: cong meng <mcpacino@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
This commit is contained in:
zees-dev 2021-09-10 14:06:57 +12:00 committed by GitHub
parent e86a586651
commit 2a60b8fcdf
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
89 changed files with 3055 additions and 139 deletions

View file

@ -0,0 +1,104 @@
package kubernetes
import (
"crypto/x509"
"encoding/base64"
"encoding/pem"
"fmt"
"io/ioutil"
"log"
"github.com/pkg/errors"
portainer "github.com/portainer/portainer/api"
)
// KubeConfigService represents a service that is responsible for handling kubeconfig operations
type KubeConfigService interface {
IsSecure() bool
GetKubeConfigInternal(endpointId portainer.EndpointID, authToken string) kubernetesClusterAccess
}
// KubernetesClusterAccess represents core details which can be used to generate KubeConfig file/data
type kubernetesClusterAccess struct {
ClusterServerURL string `example:"https://mycompany.k8s.com"`
CertificateAuthorityFile string `example:"/data/tls/localhost.crt"`
CertificateAuthorityData string `example:"MIIC5TCCAc2gAwIBAgIJAJ+...+xuhOaFXwQ=="`
AuthToken string `example:"ey..."`
}
type kubeConfigCAService struct {
httpsBindAddr string
certificateAuthorityFile string
certificateAuthorityData string
}
var (
errTLSCertNotProvided = errors.New("tls cert path not provided")
errTLSCertFileMissing = errors.New("missing tls cert file")
errTLSCertIncorrectType = errors.New("incorrect tls cert type")
errTLSCertValidation = errors.New("failed to parse tls certificate")
)
// NewKubeConfigCAService encapsulates generation of core KubeConfig data
func NewKubeConfigCAService(httpsBindAddr string, tlsCertPath string) KubeConfigService {
certificateAuthorityData, err := getCertificateAuthorityData(tlsCertPath)
if err != nil {
log.Printf("[DEBUG] [internal,kubeconfig] [message: %s, generated KubeConfig will be insecure]", err.Error())
}
return &kubeConfigCAService{
httpsBindAddr: httpsBindAddr,
certificateAuthorityFile: tlsCertPath,
certificateAuthorityData: certificateAuthorityData,
}
}
// getCertificateAuthorityData reads tls certificate from supplied path and verifies the tls certificate
// then returns content (string) of the certificate within `-----BEGIN CERTIFICATE-----` and `-----END CERTIFICATE-----`
func getCertificateAuthorityData(tlsCertPath string) (string, error) {
if tlsCertPath == "" {
return "", errTLSCertNotProvided
}
data, err := ioutil.ReadFile(tlsCertPath)
if err != nil {
return "", errors.Wrap(errTLSCertFileMissing, err.Error())
}
block, _ := pem.Decode(data)
if block == nil || block.Type != "CERTIFICATE" {
return "", errTLSCertIncorrectType
}
certificate, err := x509.ParseCertificate(block.Bytes)
if err != nil {
return "", errors.Wrap(errTLSCertValidation, err.Error())
}
return base64.StdEncoding.EncodeToString(certificate.Raw), nil
}
// IsSecure specifies whether generated KubeConfig structs from the service will not have `insecure-skip-tls-verify: true`
// this is based on the fact that we can successfully extract `certificateAuthorityData` from
// certificate file at `tlsCertPath`. If we can successfully extract `certificateAuthorityData`,
// then this will be used as `certificate-authority-data` attribute in a generated KubeConfig.
func (kccas *kubeConfigCAService) IsSecure() bool {
return kccas.certificateAuthorityData != ""
}
// GetKubeConfigInternal returns K8s cluster access details for the specified endpoint.
// On startup, portainer generates a certificate against localhost at specified `httpsBindAddr` port, hence
// the kubeconfig generated should only be utilised by internal portainer binaries as the `ClusterServerURL`
// points to the internally accessible `https` based `localhost` address.
// The struct can be used to:
// - generate a kubeconfig file
// - pass down params to binaries
func (kccas *kubeConfigCAService) GetKubeConfigInternal(endpointId portainer.EndpointID, authToken string) kubernetesClusterAccess {
clusterServerUrl := fmt.Sprintf("https://localhost%s/api/endpoints/%s/kubernetes", kccas.httpsBindAddr, fmt.Sprint(endpointId))
return kubernetesClusterAccess{
ClusterServerURL: clusterServerUrl,
CertificateAuthorityFile: kccas.certificateAuthorityFile,
CertificateAuthorityData: kccas.certificateAuthorityData,
AuthToken: authToken,
}
}

View file

@ -0,0 +1,149 @@
package kubernetes
import (
"fmt"
"io/ioutil"
"os"
"strings"
"testing"
"github.com/stretchr/testify/assert"
)
// TLS certificate can be generated using:
// openssl req -x509 -out localhost.crt -keyout localhost.key -newkey rsa:2048 -nodes -sha25 -subj '/CN=localhost' -extensions EXT -config <( \
// printf "[dn]\nCN=localhost\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:localhost\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth")
const certData = `-----BEGIN CERTIFICATE-----
MIIC5TCCAc2gAwIBAgIJAJ+poiEBdsplMA0GCSqGSIb3DQEBCwUAMBQxEjAQBgNV
BAMMCWxvY2FsaG9zdDAeFw0yMTA4MDQwNDM0MTZaFw0yMTA5MDMwNDM0MTZaMBQx
EjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBAKQ0HStP34FY/lSDIfMG9MV/lKNUkiLZcMXepbyhPit4ND/w9kOA4WTJ+oP0
B2IYklRvLkneZOfQiPweGAPwZl3CjwII6gL6NCkhcXXAJ4JQ9duL5Q6pL//95Ocv
X+qMTssyS1DcH88F6v+gifACLpvG86G9V0DeSGS2fqqfOJngrOCgum1DsWi3Xsew
B3A7GkPRjYmckU3t4iHgcMb+6lGQAxtnllSM9DpqGnjXRs4mnQHKgufaeW5nvHXi
oa5l0aHIhN6MQS99QwKwfml7UtWAYhSJksMrrTovB6rThYpp2ID/iU9MGfkpxubT
oA6scv8alFa8Bo+NEKo255dxsScCAwEAAaM6MDgwFAYDVR0RBA0wC4IJbG9jYWxo
b3N0MAsGA1UdDwQEAwIHgDATBgNVHSUEDDAKBggrBgEFBQcDATANBgkqhkiG9w0B
AQsFAAOCAQEALFBHW/r79KOj5bhoDtHs8h/ESAlD5DJI/kzc1RajA8AuWPsaagG/
S0Bqiq2ApMA6Tr3t9An8peaLCaUapWw59kyQcwwPXm9vxhEEfoBRtk8po8XblsUS
Q5Ku07ycSg5NBGEW2rCLsvjQFuQiAt8sW4jGCCN+ph/GQF9XC8ir+ssiqiMEkbm/
JaK7sTi5kZ/GsSK8bJ+9N/ztoFr89YYEWjjOuIS3HNMdBcuQXIel7siEFdNjbzMo
iuViiuhTPJkxKOzCmv52cxf15B0/+cgcImoX4zc9Z0NxKthBmIe00ojexE0ZBOFi
4PxB7Ou6y/c9OvJb7gJv3z08+xuhOaFXwQ==
-----END CERTIFICATE-----
`
// string within the `-----BEGIN CERTIFICATE-----` and `-----END CERTIFICATE-----` without linebreaks
const certDataString = "MIIC5TCCAc2gAwIBAgIJAJ+poiEBdsplMA0GCSqGSIb3DQEBCwUAMBQxEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0yMTA4MDQwNDM0MTZaFw0yMTA5MDMwNDM0MTZaMBQxEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKQ0HStP34FY/lSDIfMG9MV/lKNUkiLZcMXepbyhPit4ND/w9kOA4WTJ+oP0B2IYklRvLkneZOfQiPweGAPwZl3CjwII6gL6NCkhcXXAJ4JQ9duL5Q6pL//95OcvX+qMTssyS1DcH88F6v+gifACLpvG86G9V0DeSGS2fqqfOJngrOCgum1DsWi3XsewB3A7GkPRjYmckU3t4iHgcMb+6lGQAxtnllSM9DpqGnjXRs4mnQHKgufaeW5nvHXioa5l0aHIhN6MQS99QwKwfml7UtWAYhSJksMrrTovB6rThYpp2ID/iU9MGfkpxubToA6scv8alFa8Bo+NEKo255dxsScCAwEAAaM6MDgwFAYDVR0RBA0wC4IJbG9jYWxob3N0MAsGA1UdDwQEAwIHgDATBgNVHSUEDDAKBggrBgEFBQcDATANBgkqhkiG9w0BAQsFAAOCAQEALFBHW/r79KOj5bhoDtHs8h/ESAlD5DJI/kzc1RajA8AuWPsaagG/S0Bqiq2ApMA6Tr3t9An8peaLCaUapWw59kyQcwwPXm9vxhEEfoBRtk8po8XblsUSQ5Ku07ycSg5NBGEW2rCLsvjQFuQiAt8sW4jGCCN+ph/GQF9XC8ir+ssiqiMEkbm/JaK7sTi5kZ/GsSK8bJ+9N/ztoFr89YYEWjjOuIS3HNMdBcuQXIel7siEFdNjbzMoiuViiuhTPJkxKOzCmv52cxf15B0/+cgcImoX4zc9Z0NxKthBmIe00ojexE0ZBOFi4PxB7Ou6y/c9OvJb7gJv3z08+xuhOaFXwQ=="
func createTempFile(filename, content string) (string, func()) {
tempPath, _ := ioutil.TempDir("", "temp")
filePath := fmt.Sprintf("%s/%s", tempPath, filename)
ioutil.WriteFile(filePath, []byte(content), 0644)
teardown := func() { os.RemoveAll(tempPath) }
return filePath, teardown
}
func Test_getCertificateAuthorityData(t *testing.T) {
is := assert.New(t)
t.Run("getCertificateAuthorityData fails on tls cert not provided", func(t *testing.T) {
_, err := getCertificateAuthorityData("")
is.ErrorIs(err, errTLSCertNotProvided, "getCertificateAuthorityData should fail with %w", errTLSCertNotProvided)
})
t.Run("getCertificateAuthorityData fails on tls cert provided but missing file", func(t *testing.T) {
_, err := getCertificateAuthorityData("/tmp/non-existent.crt")
is.ErrorIs(err, errTLSCertFileMissing, "getCertificateAuthorityData should fail with %w", errTLSCertFileMissing)
})
t.Run("getCertificateAuthorityData fails on tls cert provided but invalid file data", func(t *testing.T) {
filePath, teardown := createTempFile("invalid-cert.crt", "hello\ngo\n")
defer teardown()
_, err := getCertificateAuthorityData(filePath)
is.ErrorIs(err, errTLSCertIncorrectType, "getCertificateAuthorityData should fail with %w", errTLSCertIncorrectType)
})
t.Run("getCertificateAuthorityData succeeds on valid tls cert provided", func(t *testing.T) {
filePath, teardown := createTempFile("valid-cert.crt", certData)
defer teardown()
certificateAuthorityData, err := getCertificateAuthorityData(filePath)
is.NoError(err, "getCertificateAuthorityData succeed with valid cert; err=%w", errTLSCertIncorrectType)
is.Equal(certificateAuthorityData, certDataString, "returned certificateAuthorityData should be %s", certDataString)
})
}
func TestKubeConfigService_IsSecure(t *testing.T) {
is := assert.New(t)
t.Run("IsSecure should be false", func(t *testing.T) {
kcs := NewKubeConfigCAService("", "")
is.False(kcs.IsSecure(), "should be false if TLS cert not provided")
})
t.Run("IsSecure should be false", func(t *testing.T) {
filePath, teardown := createTempFile("valid-cert.crt", certData)
defer teardown()
kcs := NewKubeConfigCAService("", filePath)
is.True(kcs.IsSecure(), "should be true if valid TLS cert (path and content) provided")
})
}
func TestKubeConfigService_GetKubeConfigInternal(t *testing.T) {
is := assert.New(t)
t.Run("GetKubeConfigInternal returns localhost address", func(t *testing.T) {
kcs := NewKubeConfigCAService("", "")
clusterAccessDetails := kcs.GetKubeConfigInternal(1, "some-token")
is.True(strings.Contains(clusterAccessDetails.ClusterServerURL, "https://localhost"), "should contain localhost address")
})
t.Run("GetKubeConfigInternal contains https bind address port", func(t *testing.T) {
kcs := NewKubeConfigCAService(":1010", "")
clusterAccessDetails := kcs.GetKubeConfigInternal(1, "some-token")
is.True(strings.Contains(clusterAccessDetails.ClusterServerURL, ":1010"), "should contain bind address port")
})
t.Run("GetKubeConfigInternal contains endpoint proxy url", func(t *testing.T) {
kcs := NewKubeConfigCAService("", "")
clusterAccessDetails := kcs.GetKubeConfigInternal(100, "some-token")
is.True(strings.Contains(clusterAccessDetails.ClusterServerURL, "api/endpoints/100/kubernetes"), "should contain endpoint proxy url")
})
t.Run("GetKubeConfigInternal returns insecure cluster access config", func(t *testing.T) {
kcs := NewKubeConfigCAService("", "")
clusterAccessDetails := kcs.GetKubeConfigInternal(1, "some-token")
wantClusterAccessDetails := kubernetesClusterAccess{
ClusterServerURL: "https://localhost/api/endpoints/1/kubernetes",
AuthToken: "some-token",
CertificateAuthorityFile: "",
CertificateAuthorityData: "",
}
is.Equal(clusterAccessDetails, wantClusterAccessDetails)
})
t.Run("GetKubeConfigInternal returns secure cluster access config", func(t *testing.T) {
filePath, teardown := createTempFile("valid-cert.crt", certData)
defer teardown()
kcs := NewKubeConfigCAService("", filePath)
clusterAccessDetails := kcs.GetKubeConfigInternal(1, "some-token")
wantClusterAccessDetails := kubernetesClusterAccess{
ClusterServerURL: "https://localhost/api/endpoints/1/kubernetes",
AuthToken: "some-token",
CertificateAuthorityFile: filePath,
CertificateAuthorityData: certDataString,
}
is.Equal(clusterAccessDetails, wantClusterAccessDetails)
})
}

View file

@ -0,0 +1,48 @@
package validation
// borrowed from apimachinery@v0.17.2/pkg/util/validation/validation.go
// https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apimachinery/pkg/util/validation/validation.go
import (
"fmt"
"regexp"
)
const dns1123LabelFmt string = "[a-z0-9]([-a-z0-9]*[a-z0-9])?"
const dns1123SubdomainFmt string = dns1123LabelFmt + "(\\." + dns1123LabelFmt + ")*"
const DNS1123SubdomainMaxLength int = 253
var dns1123SubdomainRegexp = regexp.MustCompile("^" + dns1123SubdomainFmt + "$")
// IsDNS1123Subdomain tests for a string that conforms to the definition of a subdomain in DNS (RFC 1123).
func IsDNS1123Subdomain(value string) []string {
var errs []string
if len(value) > DNS1123SubdomainMaxLength {
errs = append(errs, MaxLenError(DNS1123SubdomainMaxLength))
}
if !dns1123SubdomainRegexp.MatchString(value) {
errs = append(errs, RegexError(dns1123SubdomainFmt, "example.com"))
}
return errs
}
// MaxLenError returns a string explanation of a "string too long" validation failure.
func MaxLenError(length int) string {
return fmt.Sprintf("must be no more than %d characters", length)
}
// RegexError returns a string explanation of a regex validation failure.
func RegexError(fmt string, examples ...string) string {
s := "must match the regex " + fmt
if len(examples) == 0 {
return s
}
s += " (e.g. "
for i := range examples {
if i > 0 {
s += " or "
}
s += "'" + examples[i] + "'"
}
return s + ")"
}