使用istioctl安装istio 1.4.2
在新版的Istio当中,官方已经不推荐用Helm的方式安装了(同时不支持Helm3),推荐用istioctl命令行的方式安装。所以这里记录istioctl的方式安装Istio。
环境说明
- Amazon Linux 2 AMI,本身是基于CentOS 7
- Kubernetes 1.16.4
- Istio 1.4.2
安装Istio 1.4.2
下载 istio 1.4.2
$ wget -c "https://github.com/istio/istio/releases/download/1.4.2/istio-1.4.2-linux.tar.gz" $ tar xf istio-1.4.2-linux.tar.gz $ sudo mv istio-1.4.2/bin/istioctl /usr/local/bin $ ls bin install LICENSE manifest.yaml README.md samples tools # 整体目录结构 $ tree -L 3 -F . ├── bin/ │ └── istioctl* ├── install/ │ ├── consul/ │ │ ├── consul_config/ │ │ ├── README.md │ │ └── templates/ │ ├── gcp/ │ │ ├── bootstrap/ │ │ └── README.md │ ├── kubernetes/ │ │ ├── global-default-sidecar-scope.yaml │ │ ├── helm/ │ │ ├── istio-demo.yaml │ │ ├── mesh-expansion.yaml │ │ ├── namespace.yaml │ │ ├── operator/ │ │ └── README.md │ ├── README.md │ └── tools/ ├── LICENSE ├── manifest.yaml ├── README.md ├── samples/ │ ├── bookinfo/ │ │ ├── networking/ │ │ ├── platform/ │ │ ├── policy/ │ │ ├── README.md │ │ ├── src/ │ │ ├── swagger.yaml │ │ └── telemetry/ │ ├── certs/ │ │ ├── ca-cert.pem │ │ ├── ca-key.pem │ │ ├── cert-chain.pem │ │ ├── README.md │ │ └── root-cert.pem │ ├── custom-bootstrap/ │ │ ├── custom-bootstrap.yaml │ │ ├── example-app.yaml │ │ └── README.md │ ├── external/ │ │ ├── aptget.yaml │ │ ├── github.yaml │ │ ├── pypi.yaml │ │ └── README.md │ ├── fortio/ │ │ └── stackdriver.yaml │ ├── health-check/ │ │ ├── liveness-command.yaml │ │ ├── liveness-http-same-port.yaml │ │ └── liveness-http.yaml │ ├── helloworld/ │ │ ├── helloworld-gateway.yaml │ │ ├── helloworld.yaml │ │ ├── README.md │ │ └── src/ │ ├── httpbin/ │ │ ├── httpbin-gateway.yaml │ │ ├── httpbin-nodeport.yaml │ │ ├── httpbin-vault.yaml │ │ ├── httpbin.yaml │ │ ├── policy/ │ │ ├── README.md │ │ └── sample-client/ │ ├── https/ │ │ ├── default.conf │ │ └── nginx-app.yaml │ ├── kubernetes-blog/ │ │ ├── bookinfo-ratings.yaml │ │ ├── bookinfo-reviews-v2.yaml │ │ └── bookinfo-v1.yaml │ ├── multicluster/ │ │ └── README.md │ ├── operator/ │ │ ├── pilot-advanced-override.yaml │ │ ├── pilot-k8s.yaml │ │ ├── sds-policy-off.yaml │ │ ├── sds.yaml │ │ ├── trafficManagement-namespace.yaml │ │ ├── values-global.yaml │ │ └── values-pilot.yaml │ ├── rawvm/ │ │ └── README.md │ ├── README.md │ ├── security/ │ │ └── psp/ │ ├── sleep/ │ │ ├── policy/ │ │ ├── README.md │ │ ├── sleep-vault.yaml │ │ ├── sleep.yaml │ │ └── telemetry/ │ ├── tcp-echo/ │ │ ├── README.md │ │ ├── src/ │ │ ├── tcp-echo-20-v2.yaml │ │ ├── tcp-echo-all-v1.yaml │ │ ├── tcp-echo-services.yaml │ │ └── tcp-echo.yaml │ └── websockets/ │ ├── app.yaml │ ├── README.md │ └── route.yaml └── tools/ ├── convert_RbacConfig_to_ClusterRbacConfig.sh ├── dump_kubernetes.sh ├── _istioctl ├── istioctl.bash └── packaging/ └── common/ 44 directories, 68 files
kubernetes install 目录
# kubernetes install 目录 $ tree -L 4 -F install/kubernetes/ install/kubernetes/ ├── global-default-sidecar-scope.yaml ├── helm/ # helm安装方式 │ ├── helm-service-account.yaml │ ├── istio/ │ │ ├── charts/ │ │ │ ├── certmanager/ │ │ │ ├── galley/ │ │ │ ├── gateways/ │ │ │ ├── grafana/ │ │ │ ├── istiocoredns/ │ │ │ ├── kiali/ │ │ │ ├── mixer/ │ │ │ ├── nodeagent/ │ │ │ ├── pilot/ │ │ │ ├── prometheus/ │ │ │ ├── security/ │ │ │ ├── sidecarInjectorWebhook/ │ │ │ └── tracing/ │ │ ├── Chart.yaml │ │ ├── example-values/ │ │ │ ├── README.md │ │ │ ├── values-istio-dns-cert.yaml │ │ │ ├── values-istio-example-sds-vault.yaml │ │ │ ├── values-istio-gateways.yaml │ │ │ ├── values-istio-googleca.yaml │ │ │ ├── values-istio-meshexpansion-gateways.yaml │ │ │ └── values-istio-multicluster-gateways.yaml │ │ ├── files/ │ │ │ └── injection-template.yaml │ │ ├── README.md │ │ ├── requirements.yaml │ │ ├── templates/ │ │ │ ├── _affinity.tpl │ │ │ ├── clusterrolebinding.yaml │ │ │ ├── clusterrole.yaml │ │ │ ├── configmap.yaml │ │ │ ├── endpoints.yaml │ │ │ ├── _helpers.tpl │ │ │ ├── install-custom-resources.sh.tpl │ │ │ ├── NOTES.txt │ │ │ ├── _podDisruptionBudget.tpl │ │ │ ├── serviceaccount.yaml │ │ │ ├── service.yaml │ │ │ └── sidecar-injector-configmap.yaml │ │ ├── test-values/ │ │ │ ├── README.md │ │ │ ├── values-e2e.yaml │ │ │ ├── values-istio-auth-mcp.yaml │ │ │ ├── values-istio-auth-multicluster.yaml │ │ │ ├── values-istio-auth-non-mcp.yaml │ │ │ ├── values-istio-auth-sds.yaml │ │ │ ├── values-istio-auth.yaml │ │ │ ├── values-istio-dns-cert.yaml │ │ │ ├── values-istio-mcp.yaml │ │ │ ├── values-istio-multicluster-split-horizon.yaml │ │ │ ├── values-istio-multicluster.yaml │ │ │ ├── values-istio-non-mcp.yaml │ │ │ ├── values-istio-one-namespace-auth.yaml │ │ │ ├── values-istio-one-namespace-trust-domain.yaml │ │ │ ├── values-istio-one-namespace.yaml │ │ │ └── values-istio.yaml │ │ ├── values-istio-demo.yaml │ │ ├── values-istio-minimal.yaml │ │ ├── values-istio-remote.yaml │ │ ├── values-istio-sds-auth-control-plane-auth-disabled.yaml │ │ ├── values-istio-sds-auth.yaml │ │ └── values.yaml │ ├── istio-cni/ │ │ ├── Chart.yaml │ │ ├── templates/ │ │ │ ├── istio-cni.yaml │ │ │ └── _labels.tpl │ │ ├── values_gke.yaml │ │ └── values.yaml │ ├── istio-init/ │ │ ├── Chart.yaml │ │ ├── files/ │ │ │ ├── crd-10.yaml │ │ │ ├── crd-11.yaml │ │ │ ├── crd-14.yaml │ │ │ ├── crd-certmanager-10.yaml │ │ │ └── crd-certmanager-11.yaml │ │ ├── README.md │ │ ├── templates/ │ │ │ ├── clusterrolebinding.yaml │ │ │ ├── clusterrole.yaml │ │ │ ├── configmap-crd-10.yaml │ │ │ ├── configmap-crd-11.yaml │ │ │ ├── configmap-crd-14.yaml │ │ │ ├── configmap-crd-certmanager-10.yaml │ │ │ ├── configmap-crd-certmanager-11.yaml │ │ │ ├── job-crd-10.yaml │ │ │ ├── job-crd-11.yaml │ │ │ ├── job-crd-14.yaml │ │ │ ├── job-crd-certmanager-10.yaml │ │ │ ├── job-crd-certmanager-11.yaml │ │ │ └── serviceaccount.yaml │ │ └── values.yaml │ └── README.md ├── istio-demo.yaml ├── mesh-expansion.yaml ├── namespace.yaml ├── operator/ │ ├── charts/ │ │ ├── base/ │ │ │ ├── Chart.yaml │ │ │ ├── files/ │ │ │ ├── kustomization.yaml │ │ │ ├── templates/ │ │ │ └── values.yaml │ │ ├── gateways/ │ │ │ ├── istio-egress/ │ │ │ └── istio-ingress/ │ │ ├── istio-cni/ │ │ │ ├── Chart.yaml │ │ │ ├── templates/ │ │ │ └── values.yaml │ │ ├── istio-control/ │ │ │ ├── istio-autoinject/ │ │ │ ├── istio-config/ │ │ │ └── istio-discovery/ │ │ ├── istiocoredns/ │ │ │ ├── Chart.yaml │ │ │ ├── templates/ │ │ │ └── values.yaml │ │ ├── istio-policy/ │ │ │ ├── Chart.yaml │ │ │ ├── templates/ │ │ │ └── values.yaml │ │ ├── istio-telemetry/ │ │ │ ├── grafana/ │ │ │ ├── kiali/ │ │ │ ├── mixer-telemetry/ │ │ │ ├── prometheus/ │ │ │ ├── prometheus-operator/ │ │ │ └── tracing/ │ │ └── security/ │ │ ├── certmanager/ │ │ ├── citadel/ │ │ └── nodeagent/ │ ├── deploy/ │ │ ├── clusterrole_binding.yaml │ │ ├── clusterrole.yaml │ │ ├── crds/ │ │ │ ├── istio_v1alpha2_istiocontrolplane_crd.yaml │ │ │ └── istio_v1alpha2_istiocontrolplane_cr.yaml │ │ ├── kustomization.yaml │ │ ├── namespace.yaml │ │ ├── operator.yaml │ │ ├── service_account.yaml │ │ └── service.yaml │ ├── examples/ │ │ ├── multicluster/ │ │ │ ├── values-istio-multicluster-gateways.yaml │ │ │ └── values-istio-multicluster-primary.yaml │ │ └── vm/ │ │ ├── values-istio-meshexpansion-gateways.yaml │ │ └── values-istio-meshexpansion.yaml │ ├── profiles/ │ │ ├── default.yaml │ │ ├── demo.yaml │ │ ├── minimal.yaml │ │ ├── remote.yaml │ │ └── sds.yaml │ └── versions.yaml └── README.md 60 directories, 106 files
查看istioctl
命令行帮助:
详细参考:https://istio.io/docs/reference/commands/istioctl/
$ istioctl --help Istio configuration command line utility for service operators to debug and diagnose their Istio mesh. Usage: istioctl [command] Available Commands: authn Interact with Istio authentication policies authz (authz is experimental. Use `istioctl experimental authz`) convert-ingress Convert Ingress configuration into Istio VirtualService configuration dashboard Access to Istio web UIs deregister De-registers a service instance experimental Experimental commands that may be modified or deprecated help Help about any command kube-inject Inject Envoy sidecar into Kubernetes pod resources manifest Commands related to Istio manifests profile Commands related to Istio configuration profiles proxy-config Retrieve information about proxy configuration from Envoy [kube only] proxy-status Retrieves the synchronization status of each Envoy in the mesh [kube only] register Registers a service instance (e.g. VM) joining the mesh validate Validate Istio policy and rules verify-install Verifies Istio Installation Status or performs pre-check for the cluster before Istio installation version Prints out build version information Flags: --context string The name of the kubeconfig context to use -h, --help help for istioctl -i, --istioNamespace string Istio system namespace (default "istio-system") -c, --kubeconfig string Kubernetes configuration file --log_output_level string Comma-separated minimum per-scope logging level of messages to output, in the form of <scope>:<level>,<scope>:<level>,... where scope can be one of [ads, all, analysis, attributes, authn, cacheLog, citadelClientLog, configMapController, conversions, default, googleCAClientLog, grpcAdapter, kube, kube-converter, mcp, meshconfig, model, patch, processing, rbac, resource, runtime, sdsServiceLog, secretFetcherLog, source, stsClientLog, tpath, translator, util, validation, vaultClientLog] and level can be one of [debug, info, warn, error, fatal, none] (default "default:info,validation:error,processing:error,source:error,analysis:warn") -n, --namespace string Config namespace Use "istioctl [command] --help" for more information about a command.
查看istio版本
$ istioctl version --remote -s -o yaml
clientVersion:
golang_version: go1.13.4
revision: 35eb9dc7c6e78dac5bd8c3d142bc2a4601616932-dirty
status: Modified
tag: 1.4.2
version: 1.4.2
dataPlaneVersion:
- ID: istio-ingressgateway-6d759478d8-crwg5.istio-system
IstioVersion: 1.4.2
- ID: istio-egressgateway-68f754ccdd-2bwdn.istio-system
IstioVersion: 1.4.2
meshVersion:
- Component: citadel
Info:
golang_version: go1.13.4
revision: 35eb9dc7c6e78dac5bd8c3d142bc2a4601616932
status: Clean
tag: 1.4.2
version: 1.4.2
- Component: egressgateway
Info:
golang_version: go1.13.4
revision: 35eb9dc7c6e78dac5bd8c3d142bc2a4601616932
status: Clean
tag: 1.4.2
version: 1.4.2
- Component: galley
Info:
golang_version: go1.13.4
revision: 35eb9dc7c6e78dac5bd8c3d142bc2a4601616932
status: Clean
tag: 1.4.2
version: 1.4.2
- Component: ingressgateway
Info:
golang_version: go1.13.4
revision: 35eb9dc7c6e78dac5bd8c3d142bc2a4601616932
status: Clean
tag: 1.4.2
version: 1.4.2
- Component: pilot
Info:
golang_version: go1.13.4
revision: 35eb9dc7c6e78dac5bd8c3d142bc2a4601616932
status: Clean
tag: 1.4.2
version: 1.4.2
- Component: policy
Info:
golang_version: go1.13.4
revision: 35eb9dc7c6e78dac5bd8c3d142bc2a4601616932
status: Clean
tag: 1.4.2
version: 1.4.2
- Component: sidecar-injector
Info:
golang_version: go1.13.4
revision: 35eb9dc7c6e78dac5bd8c3d142bc2a4601616932
status: Clean
tag: 1.4.2
version: 1.4.2
- Component: telemetry
Info:
golang_version: go1.13.4
revision: 35eb9dc7c6e78dac5bd8c3d142bc2a4601616932
status: Clean
tag: 1.4.2
version: 1.4.2
查看istio 内置的profile列表:
$ istioctl profile --help
The profile subcommand lists, dumps or diffs Istio configuration profiles.
Usage:
istioctl profile [command]
Available Commands:
diff Diffs two Istio configuration profiles
dump Dumps an Istio configuration profile
list Lists available Istio configuration profiles
Flags:
--dry-run Console/log output only, make no changes.
-h, --help help for profile
--logtostderr Send logs to stderr.
--verbose Verbose output.
Global Flags:
...
Use "istioctl profile [command] --help" for more information about a command.
$ istioctl profile list
Istio configuration profiles:
demo
minimal
remote
sds
default
istio内置的profile对比
参考:https://istio.io/docs/setup/additional-setup/config-profiles/
- default: enables components according to the default settings of the
IstioControlPlane
API (recommend for production deployments). You can display the default setting by running the commandistioctl profile dump
.可用于部署生产环境的Istio - demo: configuration designed to showcase Istio functionality with modest resource requirements. It is suitable to run the Bookinfo application and associated tasks. This is the configuration that is installed with the quick start instructions, but you can later customize the configuration to enable additional features if you wish to explore more advanced tasks. 只是demo,资源request的很低,安装的功能很多This profile enables high levels of tracing and access logging so it is not suitable for performance tests.
- minimal: the minimal set of components necessary to use Istio’s traffic management features. 最小化安装
- sds: similar to the default profile, but also enables Istio’s SDS (secret discovery service). This profile comes with additional authentication features enabled by default (Strict Mutual TLS). 对比default,只开启了SDS
- remote: used for configuring remote clusters of a multicluster mesh with a shared control plane configuration. 多cluster共享一个控制面部署
The components marked as X are installed within each profile:
default | demo | minimal | sds | remote | |
---|---|---|---|---|---|
Core components | |||||
istio-citadel |
X | X | X | X | |
istio-egressgateway |
X | ||||
istio-galley |
X | X | X | ||
istio-ingressgateway |
X | X | X | ||
istio-nodeagent |
X | ||||
istio-pilot |
X | X | X | X | |
istio-policy |
X | X | X | ||
istio-sidecar-injector |
X | X | X | X | |
istio-telemetry |
X | X | X | ||
Addons | |||||
grafana |
X | ||||
istio-tracing |
X | ||||
kiali |
X | ||||
prometheus |
X | X | X |
To further customize Istio and install addons, you can add one or more --set =
options in the istioctl manifest
command that you use when installing Istio. Refer to customizing the configuration for details.
可以更改默认的profile配置来安装:参考:https://istio.io/docs/setup/install/istioctl/
查看demo的profile
$ istioctl profile dump --help The dump subcommand dumps the values in an Istio configuration profile. Usage: istioctl profile dump [<profile>] [flags] Flags: -p, --config-path string The path the root of the configuration subtree to dump e.g. trafficManagement.components.pilot. By default, dump whole tree # 查看指定的tree -f, --filename string Path to file containing IstioControlPlane CustomResource --helm-values If set, dumps the Helm values that IstioControlPlaceSpec is translated to before manifests are rendered # 查看helm values部分 -h, --help help for dump Global Flags: ... # 查看内置的demo profile $ istioctl profile dump demo autoInjection: components: injector: enabled: true k8s: replicaCount: 1 strategy: rollingUpdate: maxSurge: 100% maxUnavailable: 25% enabled: true cni: enabled: false configManagement: components: galley: enabled: true k8s: replicaCount: 1 resources: requests: cpu: 100m strategy: rollingUpdate: maxSurge: 100% maxUnavailable: 25% enabled: true defaultNamespace: istio-system gateways: components: egressGateway: enabled: true k8s: hpaSpec: maxReplicas: 5 metrics: - resource: name: cpu targetAverageUtilization: 80 type: Resource minReplicas: 1 scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: istio-egressgateway resources: limits: cpu: 2000m memory: 1024Mi requests: cpu: 10m memory: 40Mi strategy: rollingUpdate: maxSurge: 100% maxUnavailable: 25% ingressGateway: enabled: true k8s: hpaSpec: maxReplicas: 5 metrics: - resource: name: cpu targetAverageUtilization: 80 type: Resource minReplicas: 1 scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: istio-ingressgateway resources: limits: cpu: 2000m memory: 1024Mi requests: cpu: 10m memory: 40Mi strategy: rollingUpdate: maxSurge: 100% maxUnavailable: 25% enabled: true hub: docker.io/istio policy: components: policy: enabled: true k8s: env: - name: POD_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace hpaSpec: maxReplicas: 5 metrics: - resource: name: cpu targetAverageUtilization: 80 type: Resource minReplicas: 1 scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: istio-policy resources: requests: cpu: 10m memory: 100Mi strategy: rollingUpdate: maxSurge: 100% maxUnavailable: 25% enabled: true security: components: certManager: enabled: false citadel: enabled: true k8s: strategy: rollingUpdate: maxSurge: 100% maxUnavailable: 25% nodeAgent: enabled: false enabled: true tag: 1.4.2 telemetry: components: telemetry: enabled: true k8s: env: - name: POD_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace - name: GOMAXPROCS value: "6" hpaSpec: maxReplicas: 5 metrics: - resource: name: cpu targetAverageUtilization: 80 type: Resource minReplicas: 1 scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: istio-telemetry replicaCount: 1 resources: limits: cpu: 4800m memory: 4G requests: cpu: 50m memory: 100Mi strategy: rollingUpdate: maxSurge: 100% maxUnavailable: 25% enabled: true trafficManagement: components: pilot: enabled: true k8s: env: - name: POD_NAME valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace - name: GODEBUG value: gctrace=1 - name: PILOT_TRACE_SAMPLING value: "100" - name: CONFIG_NAMESPACE value: istio-config hpaSpec: maxReplicas: 5 metrics: - resource: name: cpu targetAverageUtilization: 80 type: Resource minReplicas: 1 scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: istio-pilot readinessProbe: httpGet: path: /ready port: 8080 initialDelaySeconds: 5 periodSeconds: 30 timeoutSeconds: 5 resources: requests: cpu: 10m memory: 100Mi strategy: rollingUpdate: maxSurge: 100% maxUnavailable: 25% enabled: true values: # helm values部分 certmanager: hub: quay.io/jetstack image: cert-manager-controller tag: v0.6.2 clusterResources: true galley: enableAnalysis: false image: galley gateways: istio-egressgateway: autoscaleEnabled: false env: ISTIO_META_ROUTER_MODE: sni-dnat ports: - name: http2 port: 80 - name: https port: 443 - name: tls port: 15443 targetPort: 15443 secretVolumes: - mountPath: /etc/istio/egressgateway-certs name: egressgateway-certs secretName: istio-egressgateway-certs - mountPath: /etc/istio/egressgateway-ca-certs name: egressgateway-ca-certs secretName: istio-egressgateway-ca-certs type: ClusterIP zvpn: enabled: true suffix: global istio-ingressgateway: applicationPorts: "" autoscaleEnabled: false debug: info domain: "" env: ISTIO_META_ROUTER_MODE: sni-dnat meshExpansionPorts: - name: tcp-pilot-grpc-tls port: 15011 targetPort: 15011 - name: tcp-citadel-grpc-tls port: 8060 targetPort: 8060 - name: tcp-dns-tls port: 853 targetPort: 853 ports: - name: status-port port: 15020 targetPort: 15020 - name: http2 port: 80 targetPort: 80 - name: https port: 443 - name: kiali port: 15029 targetPort: 15029 - name: prometheus port: 15030 targetPort: 15030 - name: grafana port: 15031 targetPort: 15031 - name: tracing port: 15032 targetPort: 15032 - name: tls port: 15443 targetPort: 15443 sds: enabled: false image: node-agent-k8s resources: limits: cpu: 2000m memory: 1024Mi requests: cpu: 100m memory: 128Mi secretVolumes: - mountPath: /etc/istio/ingressgateway-certs name: ingressgateway-certs secretName: istio-ingressgateway-certs - mountPath: /etc/istio/ingressgateway-ca-certs name: ingressgateway-ca-certs secretName: istio-ingressgateway-ca-certs type: LoadBalancer zvpn: enabled: true suffix: global global: arch: amd64: 2 ppc64le: 2 s390x: 2 certificates: [] configValidation: true controlPlaneSecurityEnabled: false defaultNodeSelector: {} defaultPodDisruptionBudget: enabled: true defaultResources: requests: cpu: 10m disablePolicyChecks: false enableHelmTest: false enableTracing: true imagePullPolicy: IfNotPresent imagePullSecrets: [] k8sIngress: enableHttps: false enabled: false gatewayName: ingressgateway localityLbSetting: enabled: true logAsJson: false logging: level: default:info meshExpansion: enabled: false useILB: false meshNetworks: {} mtls: auto: false enabled: false multiCluster: clusterName: "" enabled: false network: "" omitSidecarInjectorConfigMap: false oneNamespace: false operatorManageWebhooks: false outboundTrafficPolicy: mode: ALLOW_ANY policyCheckFailOpen: false priorityClassName: "" proxy: accessLogEncoding: TEXT accessLogFile: /dev/stdout accessLogFormat: "" autoInject: enabled clusterDomain: cluster.local componentLogLevel: misc:error concurrency: 2 dnsRefreshRate: 300s enableCoreDump: false envoyAccessLogService: enabled: false host: null port: null envoyMetricsService: enabled: false host: null port: null tcpKeepalive: interval: 10s probes: 3 time: 10s tlsSettings: caCertificates: null clientCertificate: null mode: DISABLE privateKey: null sni: null subjectAltNames: [] envoyStatsd: enabled: false host: null port: null excludeIPRanges: "" excludeInboundPorts: "" excludeOutboundPorts: "" image: proxyv2 includeIPRanges: '*' includeInboundPorts: '*' kubevirtInterfaces: "" logLevel: warning privileged: false protocolDetectionTimeout: 100ms readinessFailureThreshold: 30 readinessInitialDelaySeconds: 1 readinessPeriodSeconds: 2 resources: limits: cpu: 2000m memory: 1024Mi requests: cpu: 10m memory: 40Mi statusPort: 15020 tracer: zipkin proxy_init: image: proxyv2 resources: limits: cpu: 100m memory: 50Mi requests: cpu: 10m memory: 10Mi sds: enabled: false token: aud: istio-ca udsPath: "" tracer: datadog: address: $(HOST_IP):8126 lightstep: accessToken: "" address: "" cacertPath: "" secure: true zipkin: address: "" trustDomain: cluster.local useMCP: true grafana: accessMode: ReadWriteMany contextPath: /grafana dashboardProviders: dashboardproviders.yaml: apiVersion: 1 providers: - disableDeletion: false folder: istio name: istio options: path: /var/lib/grafana/dashboards/istio orgId: 1 type: file datasources: datasources.yaml: apiVersion: 1 datasources: null enabled: true env: {} envSecrets: {} image: repository: grafana/grafana tag: 6.4.3 ingress: annotations: null enabled: false hosts: - grafana.local tls: null nodeSelector: {} persist: false podAntiAffinityLabelSelector: [] podAntiAffinityTermLabelSelector: [] replicaCount: 1 security: enabled: false passphraseKey: passphrase secretName: grafana usernameKey: username service: annotations: {} externalPort: 3000 loadBalancerIP: null loadBalancerSourceRanges: null name: http type: ClusterIP storageClassName: "" tolerations: [] istiocoredns: coreDNSImage: coredns/coredns coreDNSPluginImage: istio/coredns-plugin:0.2-istio-1.1 coreDNSTag: 1.6.2 enabled: false kiali: contextPath: /kiali createDemoSecret: true dashboard: grafanaURL: null jaegerURL: null passphraseKey: passphrase secretName: kiali usernameKey: username viewOnlyMode: false enabled: true hub: quay.io/kiali ingress: annotations: null enabled: false hosts: - kiali.local tls: null nodeSelector: {} podAntiAffinityLabelSelector: [] podAntiAffinityTermLabelSelector: [] prometheusNamespace: null replicaCount: 1 security: cert_file: /kiali-cert/cert-chain.pem enabled: false private_key_file: /kiali-cert/key.pem tag: v1.9 mixer: adapters: kubernetesenv: enabled: true prometheus: enabled: true metricsExpiryDuration: 10m stackdriver: auth: apiKey: "" appCredentials: false serviceAccountPath: "" enabled: false tracer: enabled: false sampleProbability: 1 stdio: enabled: true outputAsJson: false useAdapterCRDs: false policy: adapters: kubernetesenv: enabled: true useAdapterCRDs: false autoscaleEnabled: false image: mixer sessionAffinityEnabled: false telemetry: autoscaleEnabled: false env: GOMAXPROCS: "6" image: mixer loadshedding: latencyThreshold: 100ms mode: enforce nodeSelector: {} podAntiAffinityLabelSelector: [] podAntiAffinityTermLabelSelector: [] replicaCount: 1 reportBatchMaxEntries: 100 reportBatchMaxTime: 1s sessionAffinityEnabled: false tolerations: [] useMCP: true nodeagent: image: node-agent-k8s pilot: appNamespaces: [] autoscaleEnabled: false autoscaleMax: 5 autoscaleMin: 1 configMap: true configNamespace: istio-config cpu: targetAverageUtilization: 80 deploymentLabels: null enableProtocolSniffingForInbound: false enableProtocolSniffingForOutbound: true env: {} image: pilot ingress: ingressClass: istio ingressControllerMode: "OFF" ingressService: istio-ingressgateway keepaliveMaxServerConnectionAge: 30m meshNetworks: networks: {} nodeSelector: {} podAntiAffinityLabelSelector: [] podAntiAffinityTermLabelSelector: [] policy: enabled: false replicaCount: 1 tolerations: [] traceSampling: 1 useMCP: true prometheus: contextPath: /prometheus enabled: true hub: docker.io/prom ingress: annotations: null enabled: false hosts: - prometheus.local tls: null nodeSelector: {} podAntiAffinityLabelSelector: [] podAntiAffinityTermLabelSelector: [] replicaCount: 1 retention: 6h scrapeInterval: 15s security: enabled: true tag: v2.12.0 tolerations: [] security: dnsCerts: istio-pilot-service-account.istio-control: istio-pilot.istio-control enableNamespacesByDefault: true image: citadel selfSigned: true trustDomain: cluster.local sidecarInjectorWebhook: enableNamespacesByDefault: false image: sidecar_injector injectLabel: istio-injection objectSelector: autoInject: true enabled: false rewriteAppHTTPProbe: false selfSigned: false telemetry: enabled: true v2: enabled: false tracing: enabled: true ingress: annotations: null enabled: false hosts: null tls: null jaeger: accessMode: ReadWriteMany hub: docker.io/jaegertracing memory: max_traces: 50000 persist: false spanStorageType: badger storageClassName: "" tag: "1.14" nodeSelector: {} opencensus: exporters: stackdriver: enable_tracing: true hub: docker.io/omnition resources: limits: cpu: "1" memory: 2Gi requests: cpu: 200m memory: 400Mi tag: 0.1.9 podAntiAffinityLabelSelector: [] podAntiAffinityTermLabelSelector: [] provider: jaeger service: annotations: {} externalPort: 9411 name: http-query type: ClusterIP zipkin: hub: docker.io/openzipkin javaOptsHeap: 700 maxSpans: 500000 node: cpus: 2 probeStartupDelay: 200 queryPort: 9411 resources: limits: cpu: 300m memory: 900Mi requests: cpu: 150m memory: 900Mi tag: 2.14.2 version: ""
对比default和demo的profile:
$ istioctl profile dump default > default.yaml $ istioctl profile dump demo > demo.yaml $ istioctl profile diff default.yaml demo.yaml gateways: components: egressGateway: - enabled: false + enabled: true ...
生成Kubernetes manifests yaml文件:
$ istioctl manifest generate --help
The generate subcommand generates an Istio install manifest and outputs to the console by default.
Usage:
istioctl manifest generate [flags]
Flags:
-f, --filename string Path to file containing IstioControlPlane CustomResource
--force Proceed even with validation errors
-h, --help help for generate
-o, --output string Manifest output directory path
-s, --set strings Set a value in IstioControlPlane CustomResource. e.g. --set policy.enabled=true.
Overrides the corresponding path value in the selected profile or passed through IstioControlPlane CR
customization file
Global Flags:
...
# 输出到generate-manifest-istio-yaml文件夹
$ istioctl manifest generate -o generate-manifest-istio-yaml
$ tree
.
└── generate-manifest-istio-yaml
└── Base
├── Base.yaml
├── CertManager
│ └── CertManager.yaml
├── Citadel
│ └── Citadel.yaml
├── Cni
│ └── Cni.yaml
├── EgressGateway
│ └── EgressGateway.yaml
├── Galley
│ └── Galley.yaml
├── Grafana
│ └── Grafana.yaml
├── IngressGateway
│ └── IngressGateway.yaml
├── Injector
│ └── Injector.yaml
├── Kiali
│ └── Kiali.yaml
├── NodeAgent
│ └── NodeAgent.yaml
├── Pilot
│ └── Pilot.yaml
├── Policy
│ └── Policy.yaml
├── Prometheus
│ └── Prometheus.yaml
├── PrometheusOperator
│ └── PrometheusOperator.yaml
├── Telemetry
│ └── Telemetry.yaml
└── Tracing
└── Tracing.yaml
18 directories, 17 files
# 也可以将manifest输出到一个文件
$ istioctl manifest generate > generate-manifest-istio.yaml
使用demo profile安装Istio
$ istioctl manifest apply --set profile=demo Preparing manifests for these components: - Galley - IngressGateway - NodeAgent - Citadel - PrometheusOperator - Base - CertManager - Kiali - Policy - EgressGateway - Injector - Pilot - Prometheus - Cni - CoreDNS - Grafana - Tracing - Telemetry Applying manifest for component Base Finished applying manifest for component Base Applying manifest for component Tracing Applying manifest for component Pilot Applying manifest for component Prometheus Applying manifest for component Galley Applying manifest for component Kiali Applying manifest for component Injector Applying manifest for component Policy Applying manifest for component EgressGateway Applying manifest for component IngressGateway Applying manifest for component Citadel Applying manifest for component Telemetry Applying manifest for component Grafana Finished applying manifest for component Tracing Finished applying manifest for component Citadel Finished applying manifest for component Prometheus Finished applying manifest for component Kiali Finished applying manifest for component Galley Finished applying manifest for component Injector Finished applying manifest for component Policy Finished applying manifest for component Pilot Finished applying manifest for component IngressGateway Finished applying manifest for component EgressGateway Finished applying manifest for component Grafana Finished applying manifest for component Telemetry Component Injector installed successfully: ========================================== Component Kiali installed successfully: ======================================= Component Policy installed successfully: ======================================== Component EgressGateway installed successfully: =============================================== Component Telemetry installed successfully: =========================================== Component Pilot installed successfully: ======================================= Component Prometheus installed successfully: ============================================ Component Cni installed successfully: ===================================== Component CoreDNS installed successfully: ========================================= Component Grafana installed successfully: ========================================= Component Tracing installed successfully: ========================================= Component Citadel installed successfully: ========================================= Component Galley installed successfully: ======================================== Component IngressGateway installed successfully: ================================================ Component NodeAgent installed successfully: =========================================== Component CertManager installed successfully: ============================================= Component PrometheusOperator installed successfully: ==================================================== Component Base installed successfully: ======================================
查看:
$ kubectl get all -n istio-system NAME READY STATUS RESTARTS AGE pod/grafana-6b65874977-9nlb8 1/1 Running 0 10m pod/istio-citadel-86dcf4c6b-259zj 1/1 Running 0 10m pod/istio-egressgateway-68f754ccdd-2bwdn 1/1 Running 0 10m pod/istio-galley-5fc6d6c45b-dmtb4 1/1 Running 0 10m pod/istio-ingressgateway-6d759478d8-crwg5 1/1 Running 0 10m pod/istio-pilot-5c4995d687-s8fhk 1/1 Running 0 10m pod/istio-policy-57b99968f-cfktj 1/1 Running 2 10m pod/istio-sidecar-injector-746f7c7bbb-lwgwx 1/1 Running 0 10m pod/istio-telemetry-854d8556d5-hbz4n 1/1 Running 3 10m pod/istio-tracing-c66d67cd9-z97mc 1/1 Running 0 10m pod/kiali-8559969566-rqzfs 1/1 Running 0 10m pod/prometheus-66c5887c86-44mnk 1/1 Running 0 10m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/grafana ClusterIP 10.100.239.106 <none> 3000/TCP 10m service/istio-citadel ClusterIP 10.100.238.18 <none> 8060/TCP,15014/TCP 10m service/istio-egressgateway ClusterIP 10.100.59.109 <none> 80/TCP,443/TCP,15443/TCP 10m service/istio-galley ClusterIP 10.100.173.148 <none> 443/TCP,15014/TCP,9901/TCP,15019/TCP 10m service/istio-ingressgateway LoadBalancer 10.100.112.237 <pending> 15020:31993/TCP,80:30903/TCP,443:31072/TCP,15029:32762/TCP,15030:31649/TCP,15031:32552/TCP,15032:32100/TCP,15443:31879/TCP 10m # 这里是LoadBalancer 的,我这里没有 service/istio-pilot ClusterIP 10.100.205.117 <none> 15010/TCP,15011/TCP,8080/TCP,15014/TCP 10m service/istio-policy ClusterIP 10.100.127.220 <none> 9091/TCP,15004/TCP,15014/TCP 10m service/istio-sidecar-injector ClusterIP 10.100.11.244 <none> 443/TCP 10m service/istio-telemetry ClusterIP 10.100.0.80 <none> 9091/TCP,15004/TCP,15014/TCP,42422/TCP 10m service/jaeger-agent ClusterIP None <none> 5775/UDP,6831/UDP,6832/UDP 10m service/jaeger-collector ClusterIP 10.100.63.17 <none> 14267/TCP,14268/TCP,14250/TCP 10m service/jaeger-query ClusterIP 10.100.235.187 <none> 16686/TCP 10m service/kiali ClusterIP 10.100.210.211 <none> 20001/TCP 10m service/prometheus ClusterIP 10.100.225.203 <none> 9090/TCP 10m service/tracing ClusterIP 10.100.31.32 <none> 80/TCP 10m service/zipkin ClusterIP 10.100.25.11 <none> 9411/TCP 10m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/grafana 1/1 1 1 10m deployment.apps/istio-citadel 1/1 1 1 10m deployment.apps/istio-egressgateway 1/1 1 1 10m deployment.apps/istio-galley 1/1 1 1 10m deployment.apps/istio-ingressgateway 1/1 1 1 10m deployment.apps/istio-pilot 1/1 1 1 10m deployment.apps/istio-policy 1/1 1 1 10m deployment.apps/istio-sidecar-injector 1/1 1 1 10m deployment.apps/istio-telemetry 1/1 1 1 10m deployment.apps/istio-tracing 1/1 1 1 10m deployment.apps/kiali 1/1 1 1 10m deployment.apps/prometheus 1/1 1 1 10m NAME DESIRED CURRENT READY AGE replicaset.apps/grafana-6b65874977 1 1 1 10m replicaset.apps/istio-citadel-86dcf4c6b 1 1 1 10m replicaset.apps/istio-egressgateway-68f754ccdd 1 1 1 10m replicaset.apps/istio-galley-5fc6d6c45b 1 1 1 10m replicaset.apps/istio-ingressgateway-6d759478d8 1 1 1 10m replicaset.apps/istio-pilot-5c4995d687 1 1 1 10m replicaset.apps/istio-policy-57b99968f 1 1 1 10m replicaset.apps/istio-sidecar-injector-746f7c7bbb 1 1 1 10m replicaset.apps/istio-telemetry-854d8556d5 1 1 1 10m replicaset.apps/istio-tracing-c66d67cd9 1 1 1 10m replicaset.apps/kiali-8559969566 1 1 1 10m replicaset.apps/prometheus-66c5887c86 1 1 1 10m NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE horizontalpodautoscaler.autoscaling/istio-telemetry Deployment/istio-telemetry <unknown>/80% 1 5 1 10m
注意,如上安装默认的istio-ingressgateway的Service是LoadBalancer的,我们可以在安装的时候直接更改成NodePort,如下命令:
$ istioctl manifest apply --set profile=demo --set values.gateways.istio-ingressgateway.type=NodePort
验证安装
# 生成demo profile的manifest文件: $ istioctl manifest generate --set profile=demo > generate-manifest-istio.yaml # 查看帮助 $ istioctl verify-install --help 44verify-install verifies Istio installation status against the installation file 44you specified when you installed Istio. It loops through all the installation 44resources defined in your installation file and reports whether all of them are 44in ready status. It will report failure when any of them are not ready. 44If you do not specify installation file it will perform pre-check for your cluster 44and report whether the cluster is ready for Istio installation. Usage: istioctl verify-install [flags] Examples: 44# Verify that Istio can be freshly installed 44istioctl verify-install 44 44# Verify that the deployment matches the istio-demo profile 44istioctl verify-install -f istio-demo.yaml 44 44# Verify the deployment matches a custom Istio deployment configuration 44istioctl verify-install -f $HOME/istio.yaml Flags: --enableVerbose Enable verbose output (default true) -f, --filename strings Istio YAML installation file. -h, --help help for verify-install -R, --recursive Process the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory. Global Flags: ... # 验证 $ istioctl verify-install -f generate-manifest-istio.yaml ClusterRole: istio-reader-istio-system.default checked successfully ClusterRoleBinding: istio-reader-istio-system.default checked successfully CustomResourceDefinition: attributemanifests.config.istio.io.default checked successfully CustomResourceDefinition: clusterrbacconfigs.rbac.istio.io.default checked successfully CustomResourceDefinition: destinationrules.networking.istio.io.default checked successfully CustomResourceDefinition: envoyfilters.networking.istio.io.default checked successfully CustomResourceDefinition: gateways.networking.istio.io.default checked successfully CustomResourceDefinition: httpapispecbindings.config.istio.io.default checked successfully CustomResourceDefinition: httpapispecs.config.istio.io.default checked successfully CustomResourceDefinition: meshpolicies.authentication.istio.io.default checked successfully CustomResourceDefinition: policies.authentication.istio.io.default checked successfully CustomResourceDefinition: quotaspecbindings.config.istio.io.default checked successfully CustomResourceDefinition: quotaspecs.config.istio.io.default checked successfully CustomResourceDefinition: rbacconfigs.rbac.istio.io.default checked successfully CustomResourceDefinition: rules.config.istio.io.default checked successfully CustomResourceDefinition: serviceentries.networking.istio.io.default checked successfully CustomResourceDefinition: servicerolebindings.rbac.istio.io.default checked successfully CustomResourceDefinition: serviceroles.rbac.istio.io.default checked successfully CustomResourceDefinition: virtualservices.networking.istio.io.default checked successfully CustomResourceDefinition: adapters.config.istio.io.default checked successfully CustomResourceDefinition: instances.config.istio.io.default checked successfully CustomResourceDefinition: templates.config.istio.io.default checked successfully CustomResourceDefinition: handlers.config.istio.io.default checked successfully CustomResourceDefinition: sidecars.networking.istio.io.default checked successfully CustomResourceDefinition: authorizationpolicies.security.istio.io.default checked successfully Namespace: istio-system.default checked successfully ServiceAccount: istio-reader-service-account.istio-system checked successfully ClusterRole: istio-citadel-istio-system.default checked successfully ClusterRoleBinding: istio-citadel-istio-system.default checked successfully Deployment: istio-citadel.istio-system checked successfully PodDisruptionBudget: istio-citadel.istio-system checked successfully Service: istio-citadel.istio-system checked successfully ServiceAccount: istio-citadel-service-account.istio-system checked successfully Deployment: istio-egressgateway.istio-system checked successfully PodDisruptionBudget: istio-egressgateway.istio-system checked successfully Gateway: istio-multicluster-egressgateway.istio-system checked successfully VirtualService: istio-multicluster-egressgateway.istio-system checked successfully EnvoyFilter: istio-multicluster-egressgateway.istio-system checked successfully DestinationRule: istio-multicluster-destinationrule.istio-system checked successfully Service: istio-egressgateway.istio-system checked successfully Role: istio-egressgateway-sds.istio-system checked successfully RoleBinding: istio-egressgateway-sds.istio-system checked successfully ServiceAccount: istio-egressgateway-service-account.istio-system checked successfully ClusterRole: istio-galley-istio-system.default checked successfully ClusterRoleBinding: istio-galley-admin-role-binding-istio-system.default checked successfully ConfigMap: istio-mesh-galley.istio-system checked successfully ConfigMap: istio-galley-configuration.istio-system checked successfully Deployment: istio-galley.istio-system checked successfully PodDisruptionBudget: istio-galley.istio-system checked successfully Service: istio-galley.istio-system checked successfully ServiceAccount: istio-galley-service-account.istio-system checked successfully ConfigMap: istio-grafana-configuration-dashboards-citadel-dashboard.istio-system checked successfully ConfigMap: istio-grafana-configuration-dashboards-galley-dashboard.istio-system checked successfully ConfigMap: istio-grafana-configuration-dashboards-istio-mesh-dashboard.istio-system checked successfully ConfigMap: istio-grafana-configuration-dashboards-istio-performance-dashboard.istio-system checked successfully ConfigMap: istio-grafana-configuration-dashboards-istio-service-dashboard.istio-system checked successfully ConfigMap: istio-grafana-configuration-dashboards-istio-workload-dashboard.istio-system checked successfully ConfigMap: istio-grafana-configuration-dashboards-mixer-dashboard.istio-system checked successfully ConfigMap: istio-grafana-configuration-dashboards-pilot-dashboard.istio-system checked successfully ConfigMap: istio-grafana.istio-system checked successfully Deployment: grafana.istio-system checked successfully Policy: grafana-ports-mtls-disabled.istio-system checked successfully Service: grafana.istio-system checked successfully Deployment: istio-ingressgateway.istio-system checked successfully Gateway: ingressgateway.istio-system checked successfully PodDisruptionBudget: ingressgateway.istio-system checked successfully Service: istio-ingressgateway.istio-system checked successfully ServiceAccount: istio-ingressgateway-service-account.istio-system checked successfully Sidecar: default.istio-system checked successfully ClusterRole: istio-sidecar-injector-istio-system.default checked successfully ClusterRoleBinding: istio-sidecar-injector-admin-role-binding-istio-system.default checked successfully ConfigMap: injector-mesh.istio-system checked successfully Deployment: istio-sidecar-injector.istio-system checked successfully MutatingWebhookConfiguration: istio-sidecar-injector.default checked successfully PodDisruptionBudget: istio-sidecar-injector.istio-system checked successfully Service: istio-sidecar-injector.istio-system checked successfully ServiceAccount: istio-sidecar-injector-service-account.istio-system checked successfully ConfigMap: istio-sidecar-injector.istio-system checked successfully ClusterRole: kiali.default checked successfully ClusterRole: kiali-viewer.default checked successfully ClusterRoleBinding: kiali.default checked successfully ConfigMap: kiali.istio-system checked successfully Secret: kiali.istio-system checked successfully Deployment: kiali.istio-system checked successfully Service: kiali.istio-system checked successfully ServiceAccount: kiali-service-account.istio-system checked successfully ClusterRole: istio-pilot-istio-system.default checked successfully ClusterRoleBinding: istio-pilot-istio-system.default checked successfully ConfigMap: pilot-envoy-config.istio-system checked successfully ConfigMap: istio.istio-system checked successfully Deployment: istio-pilot.istio-system checked successfully MeshPolicy: default.default checked successfully PodDisruptionBudget: istio-pilot.istio-system checked successfully Service: istio-pilot.istio-system checked successfully ServiceAccount: istio-pilot-service-account.istio-system checked successfully ClusterRole: istio-policy.default checked successfully ClusterRoleBinding: istio-policy-admin-role-binding-istio-system.default checked successfully DestinationRule: istio-policy.istio-system checked successfully Deployment: istio-policy.istio-system checked successfully PodDisruptionBudget: istio-policy.istio-system checked successfully Service: istio-policy.istio-system checked successfully ServiceAccount: istio-policy-service-account.istio-system checked successfully ClusterRole: prometheus-istio-system.default checked successfully ClusterRoleBinding: prometheus-istio-system.default checked successfully ConfigMap: prometheus.istio-system checked successfully Deployment: prometheus.istio-system checked successfully Service: prometheus.istio-system checked successfully ServiceAccount: prometheus.istio-system checked successfully HorizontalPodAutoscaler: istio-telemetry.istio-system checked successfully ClusterRole: istio-mixer-istio-system.default checked successfully ClusterRoleBinding: istio-mixer-admin-role-binding-istio-system.default checked successfully attributemanifest: istioproxy.istio-system checked successfully attributemanifest: kubernetes.istio-system checked successfully handler: stdio.istio-system checked successfully instance: accesslog.istio-system checked successfully instance: tcpaccesslog.istio-system checked successfully rule: stdio.istio-system checked successfully rule: stdiotcp.istio-system checked successfully instance: requestcount.istio-system checked successfully instance: requestduration.istio-system checked successfully instance: requestsize.istio-system checked successfully instance: responsesize.istio-system checked successfully instance: tcpbytesent.istio-system checked successfully instance: tcpbytereceived.istio-system checked successfully instance: tcpconnectionsopened.istio-system checked successfully instance: tcpconnectionsclosed.istio-system checked successfully handler: prometheus.istio-system checked successfully rule: promhttp.istio-system checked successfully rule: promtcp.istio-system checked successfully rule: promtcpconnectionopen.istio-system checked successfully rule: promtcpconnectionclosed.istio-system checked successfully handler: kubernetesenv.istio-system checked successfully rule: kubeattrgenrulerule.istio-system checked successfully rule: tcpkubeattrgenrulerule.istio-system checked successfully instance: attributes.istio-system checked successfully DestinationRule: istio-telemetry.istio-system checked successfully ConfigMap: telemetry-envoy-config.istio-system checked successfully Deployment: istio-telemetry.istio-system checked successfully PodDisruptionBudget: istio-telemetry.istio-system checked successfully Service: istio-telemetry.istio-system checked successfully ServiceAccount: istio-mixer-service-account.istio-system checked successfully Deployment: istio-tracing.istio-system checked successfully Service: jaeger-query.istio-system checked successfully Service: jaeger-collector.istio-system checked successfully Service: jaeger-agent.istio-system checked successfully Service: zipkin.istio-system checked successfully Service: tracing.istio-system checked successfully Checked 23 crds # 23个crd Checked 9 Istio Deployments # 9个Deployment,只是istio的,不包括Addons的 Istio is installed successfully
查看23个crd:
$ kubectl get crd|grep istio adapters.config.istio.io 2020-01-03T09:16:47Z attributemanifests.config.istio.io 2020-01-03T09:16:47Z authorizationpolicies.security.istio.io 2020-01-03T09:16:47Z clusterrbacconfigs.rbac.istio.io 2020-01-03T09:16:47Z destinationrules.networking.istio.io 2020-01-03T09:16:47Z envoyfilters.networking.istio.io 2020-01-03T09:16:47Z gateways.networking.istio.io 2020-01-03T09:16:47Z handlers.config.istio.io 2020-01-03T09:16:47Z httpapispecbindings.config.istio.io 2020-01-03T09:16:47Z httpapispecs.config.istio.io 2020-01-03T09:16:47Z instances.config.istio.io 2020-01-03T09:16:47Z meshpolicies.authentication.istio.io 2020-01-03T09:16:47Z policies.authentication.istio.io 2020-01-03T09:16:47Z quotaspecbindings.config.istio.io 2020-01-03T09:16:47Z quotaspecs.config.istio.io 2020-01-03T09:16:47Z rbacconfigs.rbac.istio.io 2020-01-03T09:16:47Z rules.config.istio.io 2020-01-03T09:16:47Z serviceentries.networking.istio.io 2020-01-03T09:16:47Z servicerolebindings.rbac.istio.io 2020-01-03T09:16:47Z serviceroles.rbac.istio.io 2020-01-03T09:16:47Z sidecars.networking.istio.io 2020-01-03T09:16:47Z templates.config.istio.io 2020-01-03T09:16:47Z virtualservices.networking.istio.io 2020-01-03T09:16:47Z $ kubectl get crd|grep istio|wc -l 23 # 注意api版本是不一样的 $ kubectl api-versions |grep istio authentication.istio.io/v1alpha1 config.istio.io/v1alpha2 networking.istio.io/v1alpha3 rbac.istio.io/v1alpha1 security.istio.io/v1beta1
安装实验应用 Weather forecast
项目简介
Weather Forecast 是一款查询城市的天气信息的应用示例,其展示的数据并不是真实的,只是一些静态的dummy数据,一共包含4个微服务:frontend、advertisement、forecast和recommendation。
- frontend:前台服务,会调用advertisement和forecast这两个服务,展示整个应用的页面,使用React.js开发而成。
- advertisement:广告服务,返回静态的广告图片,使用Golang开发而成。
- forecast:天气预报服务,返回相应城市的天气数据,使用Node.js开发而成。
- recommendation:推荐服务,根据天气情况向用户推荐穿衣和运动等信息,使用Java开发而成。
frontend服务有两个版本。
- v1版本的界面按钮为绿色。
- v2版本的界面按钮为蓝色。
forecast服务有两个版本。
- v1版本直接返回天气信息。
- v2版本会请求recommendation服务,获取推荐信息,并结合天气信息一起返回数据。各个服务之 间的调用关系如下图所示。
项目所需的文件:https://github.com/cloudnativebooks/cloud-native-istio
部署Weather Forecast服务
注入Sidecar
要使用Istio的功能,必须在Pod里运行Istio sidecar 代理。可以用两种方式注入:
- 通过
istioctl
命名行手动注入,此时可以修改配置。 - 通过Kubernetes的admission controller在pod创建的时候自动注入。
都会用到istio-sidecar-injector
ConfigMap。
手动注入,通过istioctl
istioctl kube-inject命令帮助:
$ istioctl kube-inject --help kube-inject manually injects the Envoy sidecar into Kubernetes workloads. Unsupported resources are left unmodified so it is safe to run kube-inject over a single file that contains multiple Service, ConfigMap, Deployment, etc. definitions for a complex application. It's best to do this when the resource is initially created. k8s.io/docs/concepts/workloads/pods/pod-overview/#pod-templates is updated for Job, DaemonSet, ReplicaSet, Pod and Deployment YAML resource documents. Support for additional pod-based resource types can be added as necessary. The Istio project is continually evolving so the Istio sidecar configuration may change unannounced. When in doubt re-run istioctl kube-inject on deployments to get the most up-to-date changes. Usage: istioctl kube-inject [flags] Examples: # Update resources on the fly before applying. kubectl apply -f <(istioctl kube-inject -f <resource.yaml>) # Create a persistent version of the deployment with Envoy sidecar # injected. istioctl kube-inject -f deployment.yaml -o deployment-injected.yaml # Update an existing deployment. kubectl get deployment -o yaml | istioctl kube-inject -f - | kubectl apply -f - # Capture cluster configuration for later use with kube-inject kubectl -n istio-system get cm istio-sidecar-injector -o jsonpath="{.data.config}" > /tmp/inj-template.tmpl kubectl -n istio-system get cm istio -o jsonpath="{.data.mesh}" > /tmp/mesh.yaml kubectl -n istio-system get cm istio-sidecar-injector -o jsonpath="{.data.values}" > /tmp/values.json # Use kube-inject based on captured configuration istioctl kube-inject -f samples/bookinfo/platform/kube/bookinfo.yaml \ 4--injectConfigFile /tmp/inj-template.tmpl \ 4--meshConfigFile /tmp/mesh.yaml \ 4--valuesFile /tmp/values.json Flags: -f, --filename string Input Kubernetes resource filename -h, --help help for kube-inject --injectConfigFile string injection configuration filename. Cannot be used with --injectConfigMapName --injectConfigMapName string ConfigMap name for Istio sidecar injection, key should be "config". (default "istio-sidecar-injector") --meshConfigFile string mesh configuration filename. Takes precedence over --meshConfigMapName if set --meshConfigMapName string ConfigMap name for Istio mesh configuration, key should be "mesh" (default "istio") -o, --output string Modified output Kubernetes resource filename --valuesFile string injection values configuration filename. Global Flags:
手动注入:
$ istioctl kube-inject -f samples/sleep/sleep.yaml | kubectl apply -f -
此时默认使用istio-sidecar-injector
ConfigMap,当然也可以将配置导出,修改,然后使用修改后的配置进行注入:
$ kubectl -n istio-system get configmap istio-sidecar-injector -o=jsonpath='{.data.config}' > inject-config.yaml $ kubectl -n istio-system get configmap istio-sidecar-injector -o=jsonpath='{.data.values}' > inject-values.yaml $ kubectl -n istio-system get configmap istio -o=jsonpath='{.data.mesh}' > mesh-config.yaml
使用上面的文件,修改,然后注入:
$ istioctl kube-inject \ --injectConfigFile inject-config.yaml \ --meshConfigFile mesh-config.yaml \ --valuesFile inject-values.yaml \ --filename samples/sleep/sleep.yaml \ | kubectl apply -f -
自动注入
创建命名空间weather,用于部署weather forecast服务,并打上标签:istio-injection=enabled
,让Istio自动注入Sidecar:
$ kubectl create ns weather $ kubectl label ns weather istio-injection=enabled
源weather-v1.yaml文件位置:cloud-native-istio/install/weather-v1.yaml
修改weather-v1.yaml,如下,主要更改了Deployment的api版本为apps/v1,并添加对应的Selector:
$ cat weather-v1.yaml ################################################################################################## # Frontend service ################################################################################################## apiVersion: v1 kind: Service metadata: name: frontend labels: app: frontend service: frontend spec: ports: - port: 3000 name: http selector: app: frontend apiVersion: apps/v1 # 修改 kind: Deployment metadata: name: frontend-v1 labels: app: frontend version: v1 spec: replicas: 1 selector: # 增加 matchLabels: app: frontend version: v1 template: metadata: labels: app: frontend version: v1 spec: containers: - name: frontend image: istioweather/frontend:v1 imagePullPolicy: IfNotPresent ports: - containerPort: 3000 ################################################################################################## # Advertisement service ################################################################################################## apiVersion: v1 kind: Service metadata: name: advertisement labels: app: advertisement service: advertisement spec: ports: - port: 3003 name: http selector: app: advertisement apiVersion: apps/v1 # 修改 kind: Deployment metadata: name: advertisement-v1 labels: app: advertisement version: v1 spec: replicas: 1 selector: # 增加 matchLabels: app: advertisement version: v1 template: metadata: labels: app: advertisement version: v1 spec: containers: - name: advertisement image: istioweather/advertisement:v1 imagePullPolicy: IfNotPresent ports: - containerPort: 3003 ################################################################################################## # Forecast service ################################################################################################## apiVersion: v1 kind: Service metadata: name: forecast labels: app: forecast service: forecast spec: ports: - port: 3002 name: http selector: app: forecast apiVersion: apps/v1 # 修改 kind: Deployment metadata: name: forecast-v1 labels: app: forecast version: v1 spec: replicas: 1 selector: # 增加 matchLabels: app: forecast version: v1 template: metadata: labels: app: forecast version: v1 spec: containers: - name: forecast image: istioweather/forecast:v1 imagePullPolicy: IfNotPresent ports: - containerPort: 3002
这个YAML文件只安装了 frontend、advertisement、forecast这3个服务的 v1版本,不包括它们的v2版本和recommendation服务。
部署:
$ kubectl -n weather apply -f weather-v1.yaml
检查部署情况:
$ kubectl -n weather get all NAME READY STATUS RESTARTS AGE pod/advertisement-v1-68d74cc5bd-hqwvz 2/2 Running 0 2m13s pod/forecast-v1-77dcd878bc-jrrpd 2/2 Running 0 2m13s pod/frontend-v1-75d4648dc6-6l4hx 2/2 Running 0 2m13s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/advertisement ClusterIP 10.100.210.127 <none> 3003/TCP 2m13s service/forecast ClusterIP 10.100.67.78 <none> 3002/TCP 2m13s service/frontend ClusterIP 10.100.89.158 <none> 3000/TCP 2m13s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/advertisement-v1 1/1 1 1 2m13s deployment.apps/forecast-v1 1/1 1 1 2m13s deployment.apps/frontend-v1 1/1 1 1 2m13s NAME DESIRED CURRENT READY AGE replicaset.apps/advertisement-v1-68d74cc5bd 1 1 1 2m13s replicaset.apps/forecast-v1-77dcd878bc 1 1 1 2m13s replicaset.apps/frontend-v1-75d4648dc6 1 1 1 2m13s
检查pod内是否有Sidecar:
$ kubectl -n weather get po frontend-v1-75d4648dc6-6l4hx -o yaml --export Flag --export has been deprecated, This flag is deprecated and will be removed in future. apiVersion: v1 kind: Pod metadata: annotations: ... labels: app: frontend security.istio.io/tlsMode: istio version: v1 spec: containers: - image: istioweather/frontend:v1 # 主程序:frontend imagePullPolicy: IfNotPresent name: frontend ports: - containerPort: 3000 protocol: TCP resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: default-token-rsfdn readOnly: true - args: - proxy - sidecar - --domain - $(POD_NAMESPACE).svc.cluster.local - --configPath - /etc/istio/proxy - --binaryPath - /usr/local/bin/envoy - --serviceCluster - frontend.$(POD_NAMESPACE) - --drainDuration - 45s - --parentShutdownDuration - 1m0s - --discoveryAddress - istio-pilot.istio-system:15010 - --zipkinAddress - zipkin.istio-system:9411 - --proxyLogLevel=warning - --proxyComponentLogLevel=misc:error - --connectTimeout - 10s - --proxyAdminPort - "15000" - --concurrency - "2" - --controlPlaneAuthPolicy - NONE - --dnsRefreshRate - 300s - --statusPort - "15020" - --applicationPorts - "3000" - --trust-domain=cluster.local env: - name: POD_NAME valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace - name: INSTANCE_IP valueFrom: fieldRef: apiVersion: v1 fieldPath: status.podIP - name: SERVICE_ACCOUNT valueFrom: fieldRef: apiVersion: v1 fieldPath: spec.serviceAccountName - name: HOST_IP valueFrom: fieldRef: apiVersion: v1 fieldPath: status.hostIP - name: ISTIO_META_POD_PORTS value: |- [ {"containerPort":3000,"protocol":"TCP"} ] - name: ISTIO_META_CLUSTER_ID value: Kubernetes - name: ISTIO_META_POD_NAME valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.name - name: ISTIO_META_CONFIG_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace - name: SDS_ENABLED value: "false" - name: ISTIO_META_INTERCEPTION_MODE value: REDIRECT - name: ISTIO_META_INCLUDE_INBOUND_PORTS value: "3000" - name: ISTIO_METAJSON_LABELS value: | {"app":"frontend","pod-template-hash":"75d4648dc6","version":"v1"} - name: ISTIO_META_WORKLOAD_NAME value: frontend-v1 - name: ISTIO_META_OWNER value: kubernetes://api/apps/v1/namespaces/weather/deployments/frontend-v1 - name: ISTIO_META_MESH_ID value: cluster.local image: docker.io/istio/proxyv2:1.4.2 #sidecar istio/proxy imagePullPolicy: IfNotPresent name: istio-proxy ports: - containerPort: 15090 name: http-envoy-prom protocol: TCP readinessProbe: failureThreshold: 30 httpGet: path: /healthz/ready port: 15020 scheme: HTTP initialDelaySeconds: 1 periodSeconds: 2 successThreshold: 1 timeoutSeconds: 1 resources: limits: cpu: "2" memory: 1Gi requests: cpu: 10m memory: 40Mi securityContext: readOnlyRootFilesystem: true runAsUser: 1337 terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /etc/istio/proxy name: istio-envoy - mountPath: /etc/certs/ name: istio-certs readOnly: true - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: default-token-rsfdn readOnly: true dnsPolicy: ClusterFirst enableServiceLinks: true initContainers: # init 容器,用于设置iptables的 - command: - istio-iptables - -p - "15001" - -z - "15006" - -u - "1337" - -m - REDIRECT - -i - '*' - -x - "" - -b - '*' - -d - "15020" image: docker.io/istio/proxyv2:1.4.2 imagePullPolicy: IfNotPresent name: istio-init resources: limits: cpu: 100m memory: 50Mi requests: cpu: 10m memory: 10Mi securityContext: capabilities: add: - NET_ADMIN runAsNonRoot: false runAsUser: 0 terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: default-token-rsfdn readOnly: true nodeName: k04.test.aws.bj.cn priority: 0 restartPolicy: Always schedulerName: default-scheduler securityContext: {} serviceAccount: default serviceAccountName: default terminationGracePeriodSeconds: 30 tolerations: - effect: NoExecute key: node.kubernetes.io/not-ready operator: Exists tolerationSeconds: 300 - effect: NoExecute key: node.kubernetes.io/unreachable operator: Exists tolerationSeconds: 300 volumes: - name: default-token-rsfdn secret: defaultMode: 420 secretName: default-token-rsfdn - emptyDir: medium: Memory name: istio-envoy - name: istio-certs secret: defaultMode: 420 optional: true secretName: istio.default
配置外部访问weather服务的frontend
使用istio-ingressgateway 80端口
文件位置:cloud-native-istio/install/weather-gateway.yaml
注意:这个文件有个BUG,apply后无法访问的,错误代码:503,正确的如下:
应用:
$ kubectl apply -f weather-gateway.yaml
配置istio ingress svc为nodePort,我这里没有Loadbalancer:
$ kubectl -n istio-system edit svc istio-ingressgateway ... spec: type: NodePort # 修改成NodePort ...
确定ingress gateways 80端口对应的node port:
# 默认已经有了80和443端口了,所以直接使用即可。
$ kubectl -n istio-system get svc
...
istio-ingressgateway NodePort 10.100.112.237 <none> 15020:31993/TCP,80:30903/TCP,443:31072/TCP,15029:32762/TCP,15030:31649/TCP,15031:32552/TCP,15032:32100/TCP,15443:31879/TCP 3d23h
...
80端口对应的NodePort是30903,所以,访问:http://x.x.x.x:30903即可,如下图:
注意:
这里提示“当前advertisement服务不可用”,F12工具查看:提示:“Chrome net::ERR_BLOCKED_BY_CLIENT”,解决方法,将浏览器的广告拦截插件关闭,我这里用的是Adblock Plus,或者不再拦截广告当前IP地址。
正常的如下:
查询天气:
使用istio-ingressgateway 非80端口
如果使用非80端口,例如8080的istio-ingressgateway,如下:
weather-gateway-8080.yaml
apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: weather-gateway-8080 namespace: istio-system spec: selector: istio: ingressgateway # use istio default controller servers: - port: number: 8080 # 使用非80端口 name: http protocol: HTTP hosts: - "*" apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: frontend-dr-8080 namespace: weather spec: host: frontend subsets: - name: v1 labels: version: v1 apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: frontend-route-8080 namespace: weather spec: hosts: - "*" gateways: - istio-system/weather-gateway-8080 http: - match: - port: 8080 route: - destination: host: frontend port: number: 3000 subset: v1 |
应用:
$ kubectl apply -f weather-gateway-8080.yaml
查看istio-ingressgateway的kubernetes service:
$ kubectl -n istio-system get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) ...
istio-ingressgateway NodePort 10.100.112.237 <none> 15020:31993/TCP,80:30903/TCP,443:31072/TCP,15029:32762/TCP,15030:31649/TCP,15031:32552/TCP,15032:32100/TCP,15443:31879/TCP 10d
...
我们发现并没有自动配置上我们配置的8080端口,但我们想看看我们配置的Gateway的是否监听了8080端口:
$ kubectl -n istio-system get po NAME READY STATUS RESTARTS AGE grafana-6b65874977-wl5ps 1/1 Running 1 3d20h istio-citadel-86dcf4c6b-9lws9 1/1 Running 2 3d20h istio-egressgateway-68f754ccdd-w4zjg 1/1 Running 1 4d20h istio-galley-5fc6d6c45b-dg729 1/1 Running 1 6d4h istio-ingressgateway-6d759478d8-gk298 1/1 Running 1 6d4h istio-pilot-5c4995d687-hv5qs 1/1 Running 1 4d20h istio-policy-57b99968f-2cpvd 1/1 Running 3 4d20h istio-sidecar-injector-746f7c7bbb-ckf4l 1/1 Running 2 4d20h istio-telemetry-854d8556d5-hbz4n 1/1 Running 6 10d istio-tracing-c66d67cd9-r6g79 1/1 Running 1 3d20h kiali-8559969566-9887z 1/1 Running 1 3d20h prometheus-66c5887c86-44mnk 1/1 Running 1 10d $ kubectl -n istio-system exec istio-ingressgateway-6d759478d8-gk298 -- netstat -lnutp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 20/envoy#这里 tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 20/envoy tcp 0 0 0.0.0.0:15090 0.0.0.0:* LISTEN 20/envoy tcp 0 0 127.0.0.1:15000 0.0.0.0:* LISTEN 20/envoy tcp6 0 0 :::15020 :::* LISTEN 1/pilot-agent
需要我们自己手动创建对应的Service资源:
weather-gateway-service-8080.yaml apiVersion: v1 kind: Service metadata: labels: app: istio-ingressgateway-8080 name: istio-ingressgateway-8080 namespace: istio-system spec: externalTrafficPolicy: Cluster ports: - name: http port: 8080 protocol: TCP targetPort: 8080 selector: app: istio-ingressgateway type: NodePort
应用,检查并测试:
$ kubectl apply -f weather-gateway-service-8080.yaml $ kubectl -n istio-system get svc ... istio-ingressgateway-8080 NodePort 10.100.3.64 <none> 8080:30112/TCP ... # 访问测试OK $ curl 10.100.3.64:8080 <!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="shortcut icon" href="/favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1,shrink-to-fit=no"/><meta name="theme-color" content="#000000"/><link rel="manifest" href="/manifest.json"/><title>天气预报</title><link href="/static/css/2.af985d11.chunk.css" rel="stylesheet"><link.....
所以结论:
当我们创建Gateway使用非80/443端口时,需要我们手动的去配置对应的Server资源。
通过Kiali 查看Weather服务调用链
更改kiali的Service类型为NodePort:
$ kubectl -n istio-system edit svc kiali ... spec: type: NodePort # 修改成NodePort ... $ kubectl -n istio-system get svc kiali NodePort 10.100.210.211 <none> 20001:32514/TCP
访问http://x.x.x.x:32514 进入Kiali:
默认用户名和密码可以通过查看Secret:
$ kubectl -n istio-system get secret kiali -o yaml apiVersion: v1 data: passphrase: YWRtaW4= username: YWRtaW4= kind: Secret metadata: annotations: ... labels: app: kiali operator.istio.io/component: Kiali operator.istio.io/managed: Reconcile operator.istio.io/version: 1.4.0 release: istio name: kiali namespace: istio-system resourceVersion: "13934" selfLink: /api/v1/namespaces/istio-system/secrets/kiali uid: cceb2838-f2c6-4273-a8a6-ead10b08cfdd type: Opaque $ echo -n YWRtaW4= |base64 -d admin
Kiali默认用户名密码都是:admin
能直接看到调用关系。如果没有显示图,可以再次访问weather或者将kiali右上角的时间选长些。