Correct date for assets in 'Kubernetes on MIPS' blog post and update blog URLs (#18663)
Taylor Dolezal
Kubernetes Prow Robot
parent
c3aea9ece8
commit
e4a09aeab3
|
|
@ -19,55 +19,55 @@ For many years, to enrich the ecology of the open-source community, we have been
|
|||
|
||||
Over the years, we have been actively participating in the Kubernetes community and have rich experience in the using and optimization of Kubernetes technology. Recently, we tried to adapt the MIPS architecture platform for Kubernetes and achieved a new a stage on that journey. The team has completed migration and adaptation of Kubernetes and related components, built not only a stable and highly available MIPS cluster but also completed the conformance test for Kubernetes v1.16.2.
|
||||
|
||||
|
||||
|
||||
|
||||
*Figure 1 Kubernetes on MIPS*
|
||||
_Figure 1 Kubernetes on MIPS_
|
||||
|
||||
## K8S-MIPS component build
|
||||
|
||||
Almost all native cloud components related to Kubernetes do not provide a MIPS version installation package or image. The prerequisite of deploying Kubernetes on the MIPS platform is to compile and build all required components on the mips64el platform. These components include:
|
||||
|
||||
- golang
|
||||
- docker-ce
|
||||
- hyperkube
|
||||
- pause
|
||||
- etcd
|
||||
- calico
|
||||
- coredns
|
||||
- metrics-server
|
||||
- golang
|
||||
- docker-ce
|
||||
- hyperkube
|
||||
- pause
|
||||
- etcd
|
||||
- calico
|
||||
- coredns
|
||||
- metrics-server
|
||||
|
||||
Thanks to the excellent design of Golang and its good support for the MIPS platform, the compilation processes of the above cloud native components are greatly simplified. First of all, we compiled Golang on the latest stable version for the mips64el platform, and then we compiled most of the above components with source code.
|
||||
|
||||
During the compilation processes, we inevitably encountered many platform compatibility problems, such as a Golang system call compatibility problem (syscall), typecasting of syscall. Stat_t from uint32 to uint64, patching for EpollEvent, and so on.
|
||||
During the compilation processes, we inevitably encountered many platform compatibility problems, such as a Golang system call compatibility problem (syscall), typecasting of syscall. Stat_t from uint32 to uint64, patching for EpollEvent, and so on.
|
||||
|
||||
To build K8S-MIPS components, we used cross-compilation technology. Our process involved integrating a QEMU tool to translate MIPS CPU instructions and modifying the build script of Kubernetes and E2E image script of Kubernetes, Hyperkube, and E2E test images on MIPS architecture.
|
||||
|
||||
After successfully building the above components, we use tools such as kubespray and kubeadm to complete kubernetes cluster construction.
|
||||
|
||||
| Name | Version | MIPS Repository |
|
||||
|------------------------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| golang on MIPS | 1.12.5 | - |
|
||||
| docker-ce on MIPS | 18.09.8 | - |
|
||||
| Name | Version | MIPS Repository |
|
||||
| ------------------------------ | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| golang on MIPS | 1.12.5 | - |
|
||||
| docker-ce on MIPS | 18.09.8 | - |
|
||||
| metrics-server for CKE on MIPS | 0.3.2 | `registry.inspurcloud.cn/library/cke/kubernetes/metrics-server-mips64el:v0.3.2` |
|
||||
| etcd for CKE on MIPS | 3.2.26 | `registry.inspurcloud.cn/library/cke/etcd/etcd-mips64el:v3.2.26` |
|
||||
| pause for CKE on MIPS | 3.1 | `registry.inspurcloud.cn/library/cke/kubernetes/pause-mips64el:3.1` |
|
||||
| hyperkube for CKE on MIPS | 1.14.3 | `registry.inspurcloud.cn/library/cke/kubernetes/hyperkube-mips64el:v1.14.3` |
|
||||
| coredns for CKE on MIPS | 1.6.5 | `registry.inspurcloud.cn/library/cke/kubernetes/coredns-mips64el:v1.6.5` |
|
||||
| coredns for CKE on MIPS | 1.6.5 | `registry.inspurcloud.cn/library/cke/kubernetes/coredns-mips64el:v1.6.5` |
|
||||
| calico for CKE on MIPS | 3.8.0 | `registry.inspurcloud.cn/library/cke/calico/cni-mips64el:v3.8.0` `registry.inspurcloud.cn/library/cke/calico/ctl-mips64el:v3.8.0` `registry.inspurcloud.cn/library/cke/calico/node-mips64el:v3.8.0` `registry.inspurcloud.cn/library/cke/calico/kube-controllers-mips64el:v3.8.0` |
|
||||
|
||||
**Note**: CKE is a Kubernetes-based cloud container engine launched by Inspur
|
||||
|
||||
|
||||
|
||||
|
||||
*Figure 2 K8S-MIPS Cluster Components*
|
||||
_Figure 2 K8S-MIPS Cluster Components_
|
||||
|
||||
|
||||
|
||||
|
||||
*Figure 3 CPU Architecture*
|
||||
_Figure 3 CPU Architecture_
|
||||
|
||||
|
||||
|
||||
|
||||
*Figure 4 Cluster Node Information*
|
||||
_Figure 4 Cluster Node Information_
|
||||
|
||||
## Run K8S Conformance Test
|
||||
|
||||
|
|
@ -81,7 +81,7 @@ Once the test has started, it launches several pods for various end-to-end tests
|
|||
|
||||
The first step is to find all needed images for the test. We can run `sonobuoy images-p e2e` command to list all images, or we can find those images in [/test/utils/image/manifest.go](https://github.com/kubernetes/kubernetes/blob/master/test/utils/image/manifest.go). Although Kubernetes officially has a complete Makefile and shell-script that provides commands for building test images, there are still a number of architecture-related issues that have not been resolved, such as the incompatibilities of base images and dependencies. So we cannot directly build mips64el architecture images by executing these commands.
|
||||
|
||||
Most test images are in golang, then compiled into binaries and built as Docker image based on the corresponding Dockerfile. These images are easy to build. But note that most images are using alpine as their base image, which does not officially support mips64el architecture for now. For this moment, we are unable to make mips64el version of [alpine] (https://www.alpinelinux.org/), so we have to replace the alpine to existing MIPS images, such as Debian-stretch, fedora, ubuntu. Replacing the base image also requires replacing the command to install the dependencies, even the version of these dependencies.
|
||||
Most test images are in golang, then compiled into binaries and built as Docker image based on the corresponding Dockerfile. These images are easy to build. But note that most images are using alpine as their base image, which does not officially support mips64el architecture for now. For this moment, we are unable to make mips64el version of [alpine](https://www.alpinelinux.org/), so we have to replace the alpine to existing MIPS images, such as Debian-stretch, fedora, ubuntu. Replacing the base image also requires replacing the command to install the dependencies, even the version of these dependencies.
|
||||
|
||||
Some images are not in `kubernetes/test/images`, such as `gcr.io/google-samples/gb-frontend:v6`. There is no clear documentation explaining where these images are locaated, though we found the source code in repository [github.com/GoogleCloudPlatform/kubernetes-engine-samples](https://github.com/GoogleCloudPlatform/kubernetes-engine-samples). We soon ran into new problems: to build these google sample images, we have to build the base image it uses, even the base image of the base images, such as `php:5-apache`, `redis`, and `perl`.
|
||||
|
||||
|
|
@ -89,44 +89,44 @@ After a long process of building an image, we finished with about four dozen ima
|
|||
|
||||
Here are some of the images we built：
|
||||
|
||||
- `docker.io/library/busybox:1.29`
|
||||
- `docker.io/library/nginx:1.14-alpine`
|
||||
- `docker.io/library/nginx:1.15-alpine`
|
||||
- `docker.io/library/perl:5.26`
|
||||
- `docker.io/library/httpd:2.4.38-alpine`
|
||||
- `docker.io/library/redis:5.0.5-alpine`
|
||||
- `gcr.io/google-containers/conformance:v1.16.2`
|
||||
- `gcr.io/google-containers/hyperkube:v1.16.2`
|
||||
- `gcr.io/google-samples/gb-frontend:v6`
|
||||
- `gcr.io/kubernetes-e2e-test-images/agnhost:2.6`
|
||||
- `gcr.io/kubernetes-e2e-test-images/apparmor-loader:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/dnsutils:1.1`
|
||||
- `gcr.io/kubernetes-e2e-test-images/echoserver:2.2`
|
||||
- `gcr.io/kubernetes-e2e-test-images/ipc-utils:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/kitten:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/metadata-concealment:1.2`
|
||||
- `gcr.io/kubernetes-e2e-test-images/mounttest-user:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/mounttest:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/nautilus:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/nonroot:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/resource-consumer-controller:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/resource-consumer:1.5`
|
||||
- `gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.10`
|
||||
- `gcr.io/kubernetes-e2e-test-images/test-webserver:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/volume/gluster:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/volume/iscsi:2.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/volume/nfs:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/volume/rbd:1.0.1`
|
||||
- `k8s.gcr.io/etcd:3.3.15`
|
||||
- `k8s.gcr.io/pause:3.1`
|
||||
- `docker.io/library/busybox:1.29`
|
||||
- `docker.io/library/nginx:1.14-alpine`
|
||||
- `docker.io/library/nginx:1.15-alpine`
|
||||
- `docker.io/library/perl:5.26`
|
||||
- `docker.io/library/httpd:2.4.38-alpine`
|
||||
- `docker.io/library/redis:5.0.5-alpine`
|
||||
- `gcr.io/google-containers/conformance:v1.16.2`
|
||||
- `gcr.io/google-containers/hyperkube:v1.16.2`
|
||||
- `gcr.io/google-samples/gb-frontend:v6`
|
||||
- `gcr.io/kubernetes-e2e-test-images/agnhost:2.6`
|
||||
- `gcr.io/kubernetes-e2e-test-images/apparmor-loader:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/dnsutils:1.1`
|
||||
- `gcr.io/kubernetes-e2e-test-images/echoserver:2.2`
|
||||
- `gcr.io/kubernetes-e2e-test-images/ipc-utils:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/kitten:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/metadata-concealment:1.2`
|
||||
- `gcr.io/kubernetes-e2e-test-images/mounttest-user:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/mounttest:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/nautilus:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/nonroot:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/resource-consumer-controller:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/resource-consumer:1.5`
|
||||
- `gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.10`
|
||||
- `gcr.io/kubernetes-e2e-test-images/test-webserver:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/volume/gluster:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/volume/iscsi:2.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/volume/nfs:1.0`
|
||||
- `gcr.io/kubernetes-e2e-test-images/volume/rbd:1.0.1`
|
||||
- `k8s.gcr.io/etcd:3.3.15`
|
||||
- `k8s.gcr.io/pause:3.1`
|
||||
|
||||
Finally, we ran the tests and got the test result, include `e2e.log`, which showed that all test cases passed. Additionally, we submitted our test result to [k8s-conformance](https://github.com/cncf/k8s-conformance) as a [pull request](https://github.com/cncf/k8s-conformance/pull/779).
|
||||
|
||||
|
||||
|
||||
|
||||
*Figure 5 Pull request for conformance test results*
|
||||
_Figure 5 Pull request for conformance test results_
|
||||
|
||||
## What's next
|
||||
|
||||
|
|
@ -136,8 +136,8 @@ In the future, we plan to actively contribute our experience and achievements to
|
|||
|
||||
Contribution plan：
|
||||
|
||||
* contribute the source of e2e test images for MIPS
|
||||
* contribute the source of hyperkube for MIPS
|
||||
* contribute the source of deploy tools like kubeadm for MIPS
|
||||
- contribute the source of e2e test images for MIPS
|
||||
- contribute the source of hyperkube for MIPS
|
||||
- contribute the source of deploy tools like kubeadm for MIPS
|
||||
|
||||
----
|
||||
---
|
||||
|
|
|
|||
|
Before Width: | Height: | Size: 212 KiB After Width: | Height: | Size: 212 KiB |
|
Before Width: | Height: | Size: 38 KiB After Width: | Height: | Size: 38 KiB |
|
Before Width: | Height: | Size: 44 KiB After Width: | Height: | Size: 44 KiB |
|
Before Width: | Height: | Size: 9.9 KiB After Width: | Height: | Size: 9.9 KiB |
|
Before Width: | Height: | Size: 216 KiB After Width: | Height: | Size: 216 KiB |
Loading…
Reference in New Issue